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ABSTRACT 


Today’s  faster  processors,  powerful  graphics  boards,  and  less-expensive  head- 
mounted  displays  have  made  virtual  reality  (VR)  a  possible  replacement  for  PC-based 
desktop  simulation.  Do  virtual  reality  systems  using  head-mounted  displays  offer 
training  advantages  to  systems  using  regular,  non-immersive  interfaces?  Specifically, 
does  a  virtual  reality  training  system  increase  retention  of  the  trained  task,  versus  a 
desktop  simulation  system?  The  retention  of  tasks  learned  in  a  virtual  reality  training 
environment  by  16  subjects  was  studied  in  comparison  to  16  subjects  in  a  desktop 
simulation  environment  in  the  context  of  a  part-task,  maintenance  training  system. 
Measures  of  training  retention  included  time  to  complete  the  task  and  procedural  errors 
observed.  Although  the  findings  indicate  that  the  difference  found  between  the 
experimental  groups  was  not  statistically  significant,  the  foundation  has  been  laid  for 
further  study  of  the  effects  of  the  increased  contextual  cues  present  in  virtual  reality 
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CHAPTER  ONE.  INTRODUCTION 


Current  personal  computer  graphics  cards  coupled  with  1  Gigahertz  and  faster 
central  processing  units  have  allowed  the  migration  of  Virtual  Reality  (VR)  training  into 
disciplines  not  previously  considered  appropriate.  In  the  area  of  maintenance  training, 
these  systems  can  allow  training  to  be  conducted  on  systems  that  are  not  physically 
available.  These  new  systems  can  also  train  mechanics  to  trouble-shoot  and  fix  problems 
on  systems  that  are  not  easily  replicated,  at  much  less  cost  than  full-scale  mock-up 
training  systems.  Current  Head-Mounted  Display  (HMD)  technology  allows  a  user  to 
wear  lightweight,  wireless  goggles  and  view  greater  than  800  x  600  resolution  images. 
These  HMDs  combined  with  a  three-dimensional  (3D)  mouse  or  a  data-glove  can  create  a 
feeling  of  immersion. 

Immersion  according  to  Witmer  and  Singer  (1998,  p.  227)  is  “a  psychological 
state  characterized  by  perceiving  oneself  to  be  enveloped  by,  included  in,  and  interacting 
with  an  environment  that  provides  a  continuous  stream  of  stimuli  and  experiences.” 
Sadowski  &  Stanney  (2002)  further  note  that  Witmer  and  Singer  suggest  that  factors  of 
immersion  include  isolation  from  the  real  world  environment,  perception  of  self-inclusion 
in  the  Virtual  Environment  (VE),  the  use  of  natural  modes  of  interaction,  and  the 
perception  of  self-movement  (p.  3).  Sadowski  &  Stanney  (2002)  state  that  other 
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researchers  including  Slater  and  Wilbur  (1997),  Draper,  Kaber,  and  Usher  (1998), 
Bystrom,  Barfield,  and  Hendrix  (1999),  believe  that  immersion  is  based  solely  on  the 
technological  aspects  of  the  system  producing  the  environment.  In  other  words,  to  be 
immersive,  a  system  must  possess  a  high-resolution  display  with  a  wide  field  of  view. 

The  system  must  also  isolate  the  user  from  real  world  sensations  such  as  light  and  sound 
to  allow  the  user  to  become  a  participant  in  the  VE,  and  not  simply  an  observer  of  it. 

Many  authors  such  as  Psotka  (1995)  believe  the  cognitive  effects  of  an  immersive 
environment  can  be  attributed  to  the  isolation  from  the  surrounding  world’s  visual, 
audible,  and  kinesthetic  sensations,  while  also  presenting  an  egocentric  experience  not 
unlike  the  real  world.  The  term  that  describes  this  sensation  in  VR  is  presence.  Presence 
is  not  simply  the  amount  of  immersion,  though.  The  degree  of  immersion  affects  the 
amount  of  presence  one  may  feel  interacting  with  the  virtual  environment  (VE),  but  an 
immersive  environment  may  not  have  a  high  level  of  presence,  if  the  VE  does  not 
resemble  the  real  world  environment  closely  enough.  Immersion  more  closely  describes 
the  physical  hardware  attributes  of  the  system,  while  presence  describes  the  cognitive 
feeling  that  “I  am  there.”  Many  authors  define  presence  as  the  perception  of  participating 
in  and  existing  within  a  virtual  environment  “in  which  one  is  immersed”  (Heeter,  1992; 
Sheridan,  1992;  Steuer,  1992;  Witmer  &  Singer,  1998).  Barfield  and  Hendrix  (1995) 
state  that  humans  can  distinguish  presence  felt  in  virtual  environments  from  that  in  the 
real  world  by  the  extent  that  participants  in  a  computer  generated  simulation  believe  they 
are  at  a  time  and  place  depicted  in  the  simulation,  versus  their  real  world  time  and 
location.  Singer  and  Witmer  (1997)  suggest  presence  is  based  on  “interactions  of  sensory 


2 


stimulation,  environmental  faetors,  and  internal  tendeneies.”  Kim  and  Bioeea  (1997) 
suggest  presenee  is  a  funetion  of  “arrival”  and  “departure”  of  experienee  in  the  VE. 
“Arrival”  involves  foeusing  on  the  VE’s  stimuli,  while  departure  is  the  shift  of  attention 
from  the  VE  stimuli  to  the  real  world.  Sadowski  and  Stanney  (2002)  follow  that 
inereasing  one’s  attention  and  foeus  on  a  VE  inereases  involvement,  thus  inereasing 
presenee. 

There  are  many  variables,  both  internal  and  external,  that  affeet  presenee. 

Stanney,  Mourant,  and  Kennedy  (1998)  and  Slater  and  Elsoh  (1993)  list  the  variables  as: 
1)  ease  of  interaetion;  2)  user- initiated  eontrol;  3)  pietorial  realism;  4)  length  of  exposure; 
5)  soeial  faetors;  and  6)  system  faetors.  Soeial  faetors  deseribe  the  effeet  that  the 
existenee  of  others  in  the  environment  inereases  the  environment’s  realism,  while  internal 
faetors  deseribe  eaeh  individual’s  pereeption  of  the  environment  (Steuer,  1992;  Heeter, 
1992).  Slater  and  Elsoh  (1993)  deseribe  system  faetors  as  the  fidelity  of  the  replieated 
environment,  the  method  of  presentation  to  the  user,  and  the  method  of  user  interaetion. 
Slater  and  Usoh  (1993)  suggest  a  positive  relationship  exists  between  the  realism  of  the 
depietion,  and  assoeiated  presenee  felt  by  the  user. 

Though  many  theorize  that  presenee  may  positively  affeet  human  performanee, 
there  exists  eonfiieting  evidenee  that  it  does  (Sadowski  and  Stanney,  2002).  None 
question  the  eaptivating  effeet  of  presenee,  though,  as  eustomers  floek  to  VR 
entertainment  deviees  at  Disney  Quest®  and  other  theme  parks  to  experienee  these 
immersive  environments.  However,  to  be  useful  in  a  training  environment,  and  to  justify 
the  added  expense  of  high-resolution  graphies,  HMDs,  and  realistie  interfaees  sueh  as 
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data  gloves,  VR  must  give  an  added  value  to  training.  Measuring  the  aetual  presenee 
generated  by  a  training  system  thus  beeomes  moot,  if  the  system  does  not  improve 
training  and  generate  added  value. 

The  existenee  of  training  transfer  from  VEs  to  the  real  world  is  not  well 
doeumented.  The  evidenee  supporting  the  existenee  of  transfer  frequently  eonflicts  with 
evidenee  supporting  its  non-existenee.  In  many  oases,  those  in  the  real  world  learn  the 
task  better  than  those  trained  in  virtual  reality  (Kozak,  Hanoook,  Arthur,  &  Chrysler, 
1993).  In  other  oases,  where  subjeots  exeouted  a  simple  spatial  task  (Rose,  Attree, 
Brooks,  Parslow,  Penn,  &  Ambihaipahan,  2000),  performed  airoraft  maintenanoe 
(Barnett,  Perrin,  Curtin  and  Helbing,  1998),  or  praotioed  using  forestry  maohinery 
(Lapointe  &  Robert,  2000),  there  was  no  signifioant  differenoe  found  between  those 
trained  using  the  real  world  equipment,  and  those  trained  in  VR. 

Although  it  is  given  that  VR  training  oan  be  effeotive,  how  does  training 
oonducted  with  desktop  simulation,  a  “poor  man’s  virtual  reality”  utilizing  a  conventional 
computer  screen,  keyboard,  and  mouse;  compare  with  training  conducted  in  virtual 
reality  using  a  head  mounted  display  and  a  3D  mouse?  The  cost  benefits  would 
obviously  be  in  favor  of  the  “poor  man’s  virtual  reality”  devices  for  some  tasks.  It  can  be 
predicted  that  both  a  desktop  simulation  system  and  a  VR  system  will  each  transfer  skills 
to  the  real  world.  However,  are  there  any  further  advantages  to  using  VR?  One  possible 
benefit  from  the  VR  consistent  with  the  concept  of  presence  is  that  virtual  environments 
contain  a  greater  number  of  spatial  cues  characteristic  of  the  real  world  environment  as 
compared  to  the  desktop  display.  The  spatial  arrangement  of  objects  in  a  VE  is  very 
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similar  to  the  arrangement  in  the  real  world.  The  spatial  eues  of  a  3D  world  projeeted 
onto  a  flat  display  do  not  give  the  same  sense  of  spatial  arrangement.  Do  these  cues 
present  in  VR,  but  not  in  desktop  simulation,  increase  the  comparative  effectiveness  of  a 
VR  system  over  a  desktop  simulation  system?  Under  the  ACT-R  model  (Anderson  & 
Lebiere,  1998),  increased  contextual  cues  relative  to  a  task  environment  should  add  to  the 
retention  or  strength  of  memory  traces  and  thus  affect  the  retention  of  trained  task 
knowledge. 

Consequently,  given  an  increased  number  of  task  relevant  contextual  cues 
represented  in  a  VR  environment  versus  that  of  the  desktop  display  simulation,  one 
would  predict  from  ACT-R  (Anderson  &  Lebiere,  1998),  greater  retention  for  tasks 
trained  in  VR  versus  a  desktop  panel  display  replica  of  the  task  environment.  In  the 
context  of  testing  whether  or  not  an  immersive  virtual  reality  tank  maintenance  training 
system  improves  retention  versus  a  desktop  simulation  with  a  monitor  and  mouse,  the 
following  research  question  emerges:  Is  there  a  significant  improvement  in  retention 
associated  with  upgrading  an  existing  Ml  Tank  Maintenance  Trainer  environment  to  an 
immersive,  three-dimensional  environment?  The  following  paragraphs  discuss  retention, 
virtual  environment  effects  on  cognition,  and  methodology  and  findings  from  an 
experiment  that  tested  the  effects  of  virtual  reality  on  human  retention  for  a  tank 
maintenance  task. 
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CHAPTER  TWO:  BACKGROUND  LITERATURE 


Learning  and  Retention 

To  maximize  learning,  one  must  understand  learning  itself.  Anderson  (1995) 
defines  learning  as  “the  process  by  which  relatively  permanent  changes  occur  in 
behavioral  potential  as  a  result  of  experience”  (p.  4).  Retention  refers  to  one’s  ability  to 
recall,  or  remember  information,  and  is  thus  related  to  memory.  Anderson  (1995)  defines 
memory  as  “the  relatively  permanent  record  of  the  experience  that  underlies  learning” 

(p.  5). 

While  learning  is  important,  the  ability  to  recall  the  learned  information  at  a  time 
and  place  useful  to  the  individual  is  paramount  to  training  for  a  specific  task.  The  study 
of  retention  is  also  the  study  of  forgetting.  Several  hypotheses  exist  to  explain 
forgetfulness.  They  are  the  decay  hypothesis  (Wickelgren,  1976),  the  interference 
hypothesis  (Keppel  and  Underwood,  1962),  and  the  retrieval-cue  hypothesis  (Tulving  and 
Psotka,  1971).  While  the  decay  hypothesis  states  that  memories  weaken  over  time 
(Wickelgren,  1976),  the  interference  hypothesis  asserts  that  memories  compete  against 
each  other  and  block  retrieval  of  a  specific  memory  (Keppel  and  Underwood,  1962),  and 
the  retrieval-cue  hypothesis  states  that  we  lose  the  cues  or  keys  to  the  memory  at  the  time 
of  retrieval  (Tulving  and  Psotka,  1971).  The  retrieval-cue  hypothesis  (Tulving  and 
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Psotka,  1971)  is  interesting  due  to  its  similarity  to  Anderson’s  ACT-R  model,  which 
describes  humans  accessing  memory  via  specific  contextual  cues.  Similar  to  the 
retrieval-cue  hypothesis  is  the  principle  of  encoding  specificity.  Encoding  specificity 
states  that  the  memories  are  best  retrieved  when  the  conditions  for  retrieval  are  most  like 
the  conditions  were  when  they  were  encoded  (Tulving  &  Thompson,  1971).  Contextual 
cues,  though  not  identical  to  configural  cues  of  classical  conditioning,  can  be  likened  to 
them  for  purpose  of  comparison  (Anderson,  1995).  Contextual  and  configural  cues  are 
the  combination  of  stimuli,  specific  to  the  environment  in  which  the  behavior  is  learned. 
Configural  cues  are  key  to  the  recall  of  the  information,  or  the  triggering  of  the  behavior. 
Anderson’s  (1998)  ACT-R  model  describes  activation  of  a  memory  structure  with  the 
following  equation  (Anderson  &  Lebiere,  1998): 

(1) 

./ 

Where  Ai  is  the  activation  the  memory  chunk  i,  and  represents  the  strength  of  a 
declarative  memory  element.  Bj  is  the  chunk’s  base  level  of  activation.  Base  level 
activation  is  a  function  of  the  frequency  of  usage  of  the  chunk  and  the  time  lag  between 
uses  of  the  chunk.  Wj  reflects  the  attentional  weighting  or  salience  of  an  element  of 
information  associated  with  a  declarative  chunk  (i.e.,  a  contextual  cue).  The  Sji  terms  are 
the  strength  of  the  association  of  the  contextual  cues  to  the  chunk  (Anderson  &  Lebiere, 
1998).  One  can  also  gather  that  the  more  specific  contextual  cues  present  that  relate  to  a 
chunk,  the  more  activation  the  chunk  in  memory  will  receive. 

Assuming  that  a  virtual  environment  accurately  modeled  after  a  real  environment 
possesses  an  increased  number  of  specific  contextual  cues  relative  to  the  training  task. 
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compared  to  a  desktop  simulation  of  the  same  environment,  the  training  eondueted  in 
sueh  an  immersive  environment  should  yield  better  retention  than  a  desktop  simulation  of 
the  same  environment.  Some  of  the  inereased  eontextual  eues  in  the  virtual  environment 
would  inelude  the  spatial  relationships  in  the  task  environment,  sueh  as  the  loeation  of 
items  in  the  turret  with  relation  to  eaeh  other  and  with  oneself.  Instead  of  looking  at  a  flat 
pieture  of  an  environment,  the  individual  immersed  in  a  virtual  environment  ean  reaeh 
out  and  experienee  the  spatial  dimensions  of  the  world,  and  sense  the  spatial  relationships 
between  objeets.  Beeause  this  environment  eontains  more  real  world-like  spatial  eues, 
memories  from  this  task  environment  may  be  more  readily  aetivated  when  the  individual 
eneounters  the  real  world  task  environment.  Inereased  aetivation  should  lead  to  less  time 
on  task,  and  improved  retention  of  task  knowledge. 

Advantages  of  New  Technology 

As  powerful  eomputer  graphies  boards  beeome  less  expensive,  more  and  more 
edueators  and  trainers  are  ehoosing  virtual  reality  as  the  medium  to  present  information. 
Consumer  graphies  boards  ineorporating  NVIDIA®  GeForeed™  ehipsets  now  eompare 
very  favorably  with  high-end  Wildeat®  eards  at  a  small  fraetion  of  the  eost.  The 
appearanee  of  this  low-eost  teehnology  in  the  marketplaee  has  enabled  budget- 
eonstrained  industries  to  embraee  VR.  From  non-invasive  heart  surgery  to  euring 
phobias,  uses  of  Head  Mounted  Displays  (HMD)  and  Data  Gloves  are  multiplying 
rapidly  (Vinee,  1999). 


Seeking  to  improve  the  effieieney  of  training  meehanics  for  the  M1A2  System 
Enhaneement  Program  (SEP)  Main  Battle  Tank  and  to  eapitalize  on  emerging 
eommereial  teehnologies,  the  El.S.  Army  Tank  and  Automotive  Command  (TACOM)  has 
initiated  an  experiment  to  test  whether  a  virtual,  immersive  training  system  is  more 
effeetive  in  training  meehanies  than  a  similar  VR  system  that  uses  a  non-immersive 
interfaee.  The  proposed  system,  ealled  the  Virtual  Immersive  Maintenanee  Trainer 
(VIMT),  uses  eommereial,  off-the-shelf  eomponents  to  keep  the  eost  of  eaeh  system 
down  under  $20  thousand.  The  eompany  produeing  the  VIMT,  Veridian  Information 
Solutions  (VIS),  believes  that  the  VIMT  will  improve  training  effeetiveness  for  students 
using  the  system.  In  addition,  VIS  believes  that  the  students  will  have  greater  spatial 
knowledge  of  the  M1A2  tank  when  working  on  the  aetual  system,  than  if  they  were  only 
exposed  to  a  non-immersive  desktop  simulation  system  (Veridian,  2001). 

Past  Experience  with  VR  Training 

Knerr,  Eampton,  Singer,  Witmer,  Goldberg,  Parsons,  and  Parsons  (1998)  found 
that  VR  ean  improve  spatial  awareness  in  navigational  tasks,  and  that  VR  aids  in 
aequiring  eonfiguration  knowledge.  Knerr  et  al.  (1998)  do  not  speeify  whether  or  not  the 
spatial  relationship  of  a  tank  turret  will  be  helpful  in  learning  turret  system  trouble¬ 
shooting  and  maintenanee  proeedures.  However,  the  report  does  speeify  what  types  of 
tasks  immersive  environments  are  suitable  to  train.  Knerr  et  al.  (1998)  state  that  VR 
systems  are  not  suitable  for  “. .  .tasks  that  involve  preeise  or  rapid  motor  aetivity”  (p.  54). 
Veridian’s  proposed  VIMT  does  not  require  rapid  motor  aetivity  (See  Appendix  B  for 
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task  list),  but  virtual  reality  historically  has  not  performed  well  in  tasks  that  do  not  require 
navigation  of  terrain  or  buildings. 

Although  the  ARI  report  does  not  specifically  address  the  same  type  of  tasks  that 
will  be  trained  on  the  VIMT,  Knerr  et  al.  (1998)  commented  that  virtual  reality  works 
best  training  tasks  that  require  spatial  awareness,  such  as  navigation  in  a  building  or  a 
maze.  Knerr  (2001)  states  an  example  of  a  maintenance  task  that  may  train  well  in 
virtual  reality  would  be  the  act  of  removing  a  vehicle’s  engine  from  its  engine  bay.  The 
mechanic  must  ensure  that  the  engine  clears  the  mounts  and  fenders  while  hoisting  it 
from  the  vehicle.  HMDs  aid  realism  for  this  task  by  allowing  the  user  to  change 
perspective  at  will,  by  moving  their  head  side  to  side  and  in  the  vertical  plane  to  gauge 
depth  and  clearance  between  objects  more  accurately.  Bums  and  Patrey  (2000) 
corroborate  Knerr  et  al.  (1998)  on  appropriate  training  tasks  for  immersive  VR  systems. 
Burns  and  Patrey  (2000)  compare  immersive  VR  techniques  to  non-immersive  techniques 
in  the  context  of  a  naval  at-sea  replenishment  scenario.  Results  of  the  experiment 
indicate  that  officers  using  an  HMD  gained  better  perception  of  lateral  spacing  of  the 
ships,  than  those  performing  the  task  with  a  regular  CRT  screen  interface. 

Troubleshooting  in  a  tank  turret  requires  some  spatial  awareness  of  the  location  of 
components,  but  for  the  most  part,  is  a  procedural  task.  Since  the  VIMT  does  not  train 
tasks  that  require  exact  spatial  awareness  or  navigation  skills,  the  involved  parties  stood 
uncertain  whether  the  VIMT  would  improve  training  effectiveness  over  the  desktop 
simulation,  until  research  confirmed  or  denied  it. 
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Evaluating  Training  in  Virtual  Environments 

Lathan,  Tracey,  Sebrechts,  Clawson,  and  Higgins  (2002)  discuss  various  methods 
to  measure  transfer  from  a  virtual  environment  to  the  real  task.  In  addition,  Lathan  et  al. 
(2002)  list  a  method  to  test  a  newly  developed  simulator  versus  an  existing  simulator. 
Their  Transfer  of  Training  Method  requires  two  groups,  with  the  experimental  group 
being  partieipants  trained  on  the  new  simulator,  and  the  control  group  training  on  the 
existing  system.  The  two  groups’  performanee  on  the  real  world  task  was  eompared  to 
determine  the  degree  of  improvement  (if  any)  that  the  new  simulator  yields.  Adding  a 
retention  test  following  a  period  of  inaetivity  to  the  transfer  of  training  test  above  would 
allow  the  measurement  of  retention  of  the  trained  task  on  the  real  world  system. 

Transfer  of  Training  from  Virtual  Environments 

A  few  authors  have  researehed  the  transfer  that  oecurs  from  tasks  performed  in 
Virtual  Environments  to  the  same  tasks  performed  in  the  real  world.  Kozak,  Haneoek, 
Arthur,  and  Chrysler  (1993)  found  that  real-world  training  proved  signifieantly  better  in 
transfer  than  did  virtual  reality  training  in  a  simple  piek  and  plaee  task  of  moving  five 
empty  soda  eans  to  and  from  rows  of  target  loeations  in  sequence.  The  two  target  rows 
were  separated  by  six  inehes.  In  the  ease  of  the  Kozak  et  al.  (1993)  study,  the  group  that 
trained  in  virtual  reality  did  not  perform  significantly  different  from  the  group  that  had  no 
training  at  all.  Kozak  et  al.  (1993)  also  state  that  the  laek  of  transfer  from  the  VR  may  be 
due  to  its  lack  of  fidelity  (limited  by  the  then-current  technology)  to  the  real  world 
environment.  Psotka  (1995)  attributes  the  laek  of  transfer  found  in  the  Kozak  et  al. 
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(1993)  study  to  the  simplicity  of  the  task  tested,  and  that  a  more  difficult  task  may  have 
shown  transfer. 

In  the  past  eight  years,  computer  technology  has  improved  to  allow  for  an 
increase  in  the  fidelity  of  virtual  environments.  In  more  recent  studies,  the  gap  between 
VR  training  and  real-world  training  has  closed  significantly.  As  an  example.  Rose  et  al. 
(2000)  found  no  significant  difference  in  post-training  performance  when  training  with 
VR  and  real-world  venues  on  a  steadiness  task  (i.e.,  requiring  one  to  maneuver  a  ring 
over  a  bent  wire  without  touching  the  ring  to  the  wire).  The  study  by  Rose,  et  al.  (2000) 
reinforces  the  idea  that  the  quality  of  the  simulation  and  training  conducted  in  the  VR 
trainer  affect  the  amount  of  skill  transfer  that  takes  place  between  the  artificial 
environment  and  the  real  world.  In  a  separate  study,  Barnett,  Perrin,  Curtin,  and  Helbing 
(1998)  also  concluded  that  training  motor  skills  in  VR  and  real-world  environments 
yielded  statistically  similar  results.  The  near-equality  of  VR  and  real  world  training  for 
specific  tasks  presents  a  great  advantage  for  training  expensive  or  dangerous  activities. 

Lapointe  and  Robert  (2000)  found  that  using  a  VR  Forestry  Machine  Trainer 
decreased  the  necessary  training  time  of  new  operators,  and  increased  the  initial 
productivity  output  of  the  trainees,  once  they  transitioned  to  the  actual  machine.  Their 
studies  showed  that  using  VR  for  the  initial  training  of  new  operators  reduced  repair  and 
maintenance  costs  by  26  percent,  and  increased  production  by  23  percent  during  the  new 
operators’  first  month  of  actual  work  with  the  forestry  machines. 
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The  task  of  tank  turret  troubleshooting  differs  from  the  forestry  task  studied  by 
Lapointe  and  Robert  (2000)  in  that  it  requires  less  spatial  awareness,  and  slightly  more 
proeedural  knowledge. 

In  any  ease,  to  be  effeetive,  a  VR  training  system  must  adequately  represent  the 
task  or  tasks  to  be  trained  (i.e.,  provide  suffleient  eontextual  eues  relative  to  the  task  or 
goal  at  hand),  and  provide  adequate  feedbaek  to  the  student  on  their  performanee  in  the 
training  system.  Although  the  studies  above  have  shown  that  training  on  speeifie  tasks 
eondueted  in  VR  does  transfer  positively  to  the  real  world,  this  should  not  be  a  earte 
blanehe  aeeeptanee  of  VR  for  training  tasks  in  general.  Additionally,  there  are  eost 
benefits  interaetions  that  must  be  taken  into  aeeount  when  deeiding  upon  the 
appropriateness  of  VR  for  training  speeifie  tasks.  Additional  experimentation  must  be 
performed  in  order  to  judge  whether  a  VR  model  of  an  environment  enables  a  trainee  to 
retain  and  transfer  more  knowledge  than  does  a  desktop  simulation  model  of  the  same 
environment,  espeeially  given  the  eost  differential  between  these  two  modes  of  task 
environment  simulation. 

Retention  of  Tasks  within  Virtual  Environments 

Finkelstein  (1999)  studied  the  retention  of  tasks  within  virtual  environments  and 
developed  twelve  guidelines  to  aid  in  task  retention.  Though  interesting,  Finkelstein’s 
researeh  foeused  on  the  retention  of  tasks  within  a  virtual  environment  following  a  one- 
week  delay,  versus  the  eoneern  of  this  study,  whieh  is  the  retention  of  tasks  transferred  to 
the  real  world  environment.  In  his  findings,  Finkelstein  attributes  an  inerease  in  retention 
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to  the  amount  of  “guidelines”  (landmarks,  imbedded  photos  of  movie  charaeters,  and 
other  salient  features)  ineluded  in  the  VE. 

Finkelstein’s  (1999)  finding  that  adding  “guidelines”  inereases  retention  parallel 
the  assertions  of  Anderson’s  and  Lebiere’s  (1998)  ACT-R  model,  that  notes  an  inerease 
in  retention  with  an  inerease  in  assoeiated  eontextual  eues.  Sinee  Finkelstein’s 
“guidelines”  add  sueh  eues  to  the  environment,  then  by  Anderson’s  and  Febiere’s  (1998) 
ACT-R  model,  they  should  inerease  retention  of  the  task  when  tested  outside  of  the  VR 
environment. 
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CHAPTER  THREE:  METHOD 


Research  Question:  Is  there  a  significant  improvement  in  retention  associated 
with  upgrading  the  existing  M1A2  System  Enhancement  Program  (SEP)  Tank 
Maintenance  Trainer  environment  to  an  immersive,  three-dimensional  environment? 

In  order  to  answer  the  above  research  question,  an  experiment  was  conducted  to 
test  whether  or  not  using  the  three-dimensional  immersive  environment  of  the  Virtual 
Immersive  Maintenance  Trainer  increases  retention  in  subjects,  in  comparison  to  those 
subjects  using  the  desktop  simulation  version  of  the  trainer. 

Task 

The  training  systems  compared,  train  diagnosing  and  trouble-shooting  turret 
malfunctions  in  a  Ml  A2-SEP  Tank.  The  tasks  selected  for  the  experiment  was  the 
Weapon  System  Internal  Eault  Isolation  Test  (InterEIT).  The  InterEIT  is  a  relatively 
stationary  task  inside  the  tank  turret  requiring  the  reading  of  displays  and  manipulation  of 
controls.  The  Weapon  System  InterEIT  was  chosen  because  it  requires  the  mechanic  to 
traverse  the  space  of  the  turret  repeatedly  to  manipulate  controls  and  check  the  status  of 
lights  and  displays.  The  task  does  not  require  rapid  head  or  body  movement,  though  the 
subject  must  operate  and  view  devices  from  many  viewpoints  and  positions  within  the 
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turret.  It  is  a  generally  a  proeedural  task  requiring  some  knowledge  of  the  spatial 
arrangement  of  the  eomponents  of  the  tank  turret. 

The  desktop  simulation  subjeets  performed  the  two  tasks  while  using  the  desktop 
diagnostie/troubleshooting  trainer.  They  aetivated  switehes  and  buttons  using  a  point- 
and-elick  interfaee  via  a  Mierosoft®  Wheel  Mouse.  In  order  to  use  a  tool  to  remove  a 
part,  the  soldier  clicks  on  a  tool  icon,  which  turns  his  cursor  into  a  wrench,  and  then 
allows  him  to  remove  or  replace  selected  parts  with  a  single  mouse  click.  The  desktop 
subjects  change  the  view  of  the  interior  of  the  turret  by  clicking  on  graphical  buttons 
displayed  on  a  toolbar  at  the  bottom  of  the  screen.  Each  direction  of  movement  or 
rotation  (six  degrees  of  freedom)  is  represented  by  its  own  arrow  icon. 

The  VR  subjects  performed  the  task  on  the  VIMT,  using  a  point-and-click 
interface  to  activate  switches  and  buttons  via  a  Polhemus®  3Ball  3D  mouse.  The  VR 
subjects  changed  their  view  of  the  turret  by  moving  or  rotating  their  head  in  the  desired 
direction.  The  VIMT  tracks  head  movement  in  six  degrees  of  freedom.  The  VIMT 
provides  an  immersive  3D  representation  of  the  tank  turret  environment  versus  the 
desktop  flat  panel  display  portraying  the  same  equipment  information  in  the  absence  of 
holistic  configuration  of  turret  spatial  relations. 

Subjects 

The  32  subjects  selected  for  the  experiment  were  all  students  in  a  United  States 
Army  M1A2  SEP  tank  mechanic  training  course.  They  are  all  enlisted  soldiers  in 
Military  Occupational  Specialty  63 A  (Abrams  Tank  Systems  Mechanic),  ranging  in  age 
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from  18  to  32  years,  with  an  average  age  of  20.9  years  (See  Table  G-1  in  Appendix  G). 
All  of  the  subjeets  are  male.  Nineteen  subjeets  are  Caueasian,  eight  are  Hispanie,  and 
five  are  black.  All  of  the  subjects  have  a  high  school  level  education  or  better.  All  of  the 
subjects,  except  one,  were  new  enlistees  who  have  been  training  on  Ml  tanks  for  4 
months,  and  have  been  selected  to  receive  training  on  the  newest  version  of  the  Ml  tank, 
the  M1A2  SEP.  The  one  exception  is  a  fourteen-year  veteran  tank  mechanic  previously 
untrained  on  the  M1A2  SEP  tank,  but  very  experienced  on  the  older  versions  of  the 
armored  vehicle. 

The  class  was  divided  evenly  by  their  grades  to  date  in  their  ten-month  training 
program,  placing  equal  numbers  of  high  (above  85  %  test  scores)  and  low  scoring 
students  (several  test  scores  below  70  %)  in  each  experimental  group.  The  veteran 
mechanic  was  considered  a  high  scoring  student,  and  was  placed  in  the  experimental 
group. 


Materials 

The  two  different  devices  used  in  the  experiment  were  the  existing  M1A2  SEP 
Diagnostic  Troubleshooting  trainer  (See  Eigure  1),  and  the  Virtual  Immersive 
Maintenance  Trainer  prototype  (See  Eigure  2). 
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Figure  1.  M1A2  SEP  Diagnostic  Trainer  (Desktop  Simulation) 


The  desktop  simulation  trainer  uses  1280  x  1024  graphic  resolution  produced  by  a 
Silicon  Graphics  workstation  displayed  on  two  monitors  (See  Table  1  for  system 
specifications).  The  right  monitor  (See  Figure  1)  displays  a  three-dimensional  view  of 
the  inside  of  the  tank  turret,  while  the  left  display  presents  a  detailed,  two-dimensional 
view  of  a  user-selected  computer  display  screen  in  the  turret.  Unlike  in  the  real  tank,  the 
three-dimensional  (right-hand  screen)  view  on  the  desktop  simulation  system  does  not 
depict  data  that  would  be  presented  on  the  vehicle’s  computer  displays  in  the  real  tank. 

To  see  data  on  the  tank  computer  screens,  the  user  must  view  the  left  CRT  display  (2D 
view).  The  user  interface  for  the  desktop  simulation  system  is  a  conventional  mouse  and 
a  101-key  keyboard. 
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Table  1.  M1A2  SEP  Desktop  Simulation  Trainer  System  Specifieations 
System 


Proeessors 

System  Memory 

Operating  System 

Graphies  Proeessor 

Resolution 

Dual  866MHz  Pentium®!!! 

1.5  GBRAM(SD133) 

MS  Windows®  2000 

!ntense3D®  Wildcat®  4210 

1280  X  1024 

Display 

Manufacturer 

CTX® 

Model 

Two  VL950  CRT  Monitors  (19  in) 

Pitch 

0.26  mm 

Mouse 

Manufacturer 

Microsoft™ 

Model 

Intellimouse®  with  wheel 

The  VIMT  prototype  (See  Figure  2)  uses  a  Cy-visor™  DH-4400VP  HMD  with 
800  X  600  resolution  and  a  3 1  degree  diagonal  field-of-view  (25  degree  Horizontal  field- 
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Figure  2.  M1A2  SEP  Virtual  Immersive  Maintenanee  Trainer 


of-view).  The  DH-4400VP  allows  adjustment  of  its  screens’  inter-pupillary  distance 
(IPD).  It  does  not  possess  diopter  adjustments.  These  3D  goggles  do  accommodate  the 
wear  of  eyeglasses  through  interchangeable  eyecups,  which  allow  additional  clearance 
with  the  user’s  face.  Although  the  HMD  used  has  a  relatively  small  field-of-view,  its 
dual  displays  contain  1 .44  million  pixels  each,  allowing  sufficient  resolution  to  read  all 
labels  and  displays  present  in  the  virtual  environment.  The  virtual  reality  trainer  uses  a 
three-dimensional  mouse  as  its  interface,  and  navigation  in  the  VE  is  performed  by  the 
subject  rotating  or  moving  his  head  with  6  degrees-of-freedom.  Additionally,  a 
conventional  computer  monitor  is  present  to  allow  an  instructor  or  observer  to  track  what 
the  user  is  viewing  in  the  three-dimensional  space  (See  Eigure  2).  Unlike  the  desktop 
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simulation  system,  which  uses  two  screens,  the  VIMT’s  graphics  depict  the  tank’s 
computer  screens  as  they  would  appear  in  the  real  tank,  complete  with  textual  and 
graphical  data.  See  Table  2  (next  page)  for  the  VIMT  system’s  specifications. 
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Table  2.  VIMT  System  Specifieations 


System 

Proeessor 

System  Memory 

Operating  System 

Graphies  Proeessor 

Resolution 

Dual  AMD  AthlonTM  L533MHz 

785.9  MB  RAM 

MS  Windows®  2000,  Ver.  5.002195  (SP2) 
NVIDIATM  Geforce3®  Ti  500 

800  X  600 

Display  (HMD) 

Manufacturer 

Personal  Display  Systems,  Inc. 

Model 

I- Visor®  DH-4400VP 

LCD 

0.49m,  1.44  million  pixels 

Dot  number  /  panel 

800  (H)  X  600  (V)  X  RGB 

Virtual  Image  Size 

44m.  at  2m 

Viewing  Angle 

31  deg.  diag.  (HFOV  25  deg.) 

PC  Input 

RGB 

Mouse  (3D) 

Manufacturer 

Polhemus® 

Model 

3Ball® 

VR  Tracker 

Manufacturer 

Polhemus® 

Model 

3  Space  Fastrack®  Receiver 

Latency 

4ms 

Update  Rate 

120Hz 

VGA  Splitter 

Manufacturer 

ATEN® 

Model 

VS  104  VGA  Splitter 
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Procedure 


The  experiment  utilized  a  two-group  design,  with  repeated  measures,  and 
included  a  retention  test  following  a  two-day  weekend  break.  The  experimental  design 
resembles  the  Transfer  of  Training  Method  (Lathan  et  ah,  2002)  discussed  earlier  in 
Chapter  2,  with  the  addition  of  the  retention  test. 

The  experiment  was  conducted  on  32  subjects  during  two  regularly  meeting 
mechanic  training  courses  in  January  and  March  2002.  The  subject’s  scores  (elapsed 
time  to  complete  the  tasks,  and  errors  made)  on  the  System  Power-up  and  the  Weapon 
System  InterFIT  were  recorded  each  training  day  (See  Table  3). 


Table  3.  Experimental  Design  Data  Collection  Schedule 

Wed.  Thurs.  Fri.  Mon. 

Task 

System  Power-up  S,  T  T  T  Re,  R,  Q 

Weapon  System  InterFIT  S,  T  T  T  Re,  R,  Q 

Note.  S  =  Spatial  Ability  Test,  T  =  Task  Training,  Re  =  Recall  Test  (Appendix  H) 
R  =  Task  Retention  Test  on  Real  Tank,  Q  =  Questionnaires  (See  next  sub-heading) 


The  control  group  subjects  were  scored  using  the  diagnostic  software  imbedded  in 
the  training  system.  The  desktop  system  tallies  subject  errors  when  they  fail  to  follow  the 
task  procedure  properly.  The  subjects  using  the  VIMT  prototype  were  manually  scored 
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by  the  experimenter  by  reeording  the  elapsed  time,  and  number  of  errors  made  exeeuting 
the  task.  Following  the  weekend  break,  the  subjeets  took  the  retention  test  on  the  real 
M1A2  SEP  tank,  without  the  use  of  the  maintenanee  manual  available  to  them  in  prior 
tests. 

The  subjeets  were  seored  on  their  elapsed  time  to  eomplete  the  task,  and 
proeedural  errors  they  made  during  their  exeeution  of  the  Weapon  System  InterFIT. 
Additionally,  eaeh  subjeet’s  seore  on  a  spatial  abilities  test  taken  prior  to  testing  was  used 
to  adjust  the  seores.  Regression  analysis  was  eondueted  on  Equation  2  (below)  to  ereate 
an  adjusted  seores  equation: 

Time  =  Bq+  B^Spatial  +  B^Errors  +  B^R  +  B^Error  x  VR  (2) 

Where  VR  is  a  qualitative  variable  that  equals  1  for  subjeets  using  the  VIMT,  and 
equals  0  for  the  desktop  simulation  subjeets. 

Following  the  adjustment  of  the  seores  aeeording  to  Equation  2,  the  adjusted 
times  were  analyzed  with  a  test  of  hypothesis  using  a  t  statistieal  test.  The  t  test  is 
appropriate  beeause  there  are  only  two  experimental  groups,  with  a  small  number  of 
subjeets.  The  t  test  assumptions  require  that  the  samples  be  randomly  seleeted  from  the 
target  groups,  and  that  the  varianees  of  the  seores  within  the  two  groups  are  equal.  The 
use  of  the  t  statistie  is  valid,  assuming  the  Army  randomly  seleets  meehanies  for 
assignment  to  M1A2  units.  In  addition,  sinee  the  meehanies  are  all  sampled  from  the 
Army,  a  single  population,  it  was  assumed  that  the  varianees  in  their  performanees  were 
equal.  The  null  and  alternative  hypotheses  are  listed  below. 
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Ho  =  There  is  no  differenee  between  adjusted  test  scores  of  the  group  of 
mechanics  using  the  VIMT,  and  the  group  using  the  desktop  simulation.  Timeoesktop- 
TimeviMT=  0. 

Ha  =  There  is  a  significant  difference  in  average  adjusted  test  scores  in  favor  of 
the  VIMT  test  group.  Timeoesktop-  TimeviMT>  0. 

Rejecting  the  null  hypothesis  would  have  indicated  that  the  VIMT  group  task 
retention  is  higher  than  the  task  retention  of  the  group  using  the  desktop  simulation 
system. 

The  average  procedural  errors  made  by  each  group  were  also  analyzed  with  a  t 
statistical  test  to  see  if  there  was  a  significant  difference  between  the  two  groups. 

In  addition,  a  t  test  would  have  been  utilized  to  examine  the  differences  between 
groups  for  the  scores  from  the  training  tests.  However,  a  comparison  of  training  trials 
between  the  experimental  groups  was  not  conducted,  due  to  the  very  large  differences  in 
elapsed  time  recorded  during  training  trials  on  the  two  systems.  This  large  difference  is 
due  to  the  VIMT  replicating  only  the  System  Power-up  and  Weapon  System  InterFIT 
tasks,  while  the  desktop  simulation  required  that  additional  tasks  be  performed  prior  to 
executing  the  Weapon  System  InterFIT.  These  additional  tasks  accounted  for  the  large 
difference  in  training  trial  elapsed  time  between  the  two  groups  of  subjects.  See  tables  in 
Appendix  G  for  the  listing  of  raw  scores. 
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Questionnaires 


In  addition  to  the  conducting  an  experimental  test  of  hypothesis,  surveys  were 
distributed  to  the  subjects.  Before  the  experiment,  each  student  completed  a  spatial 
abilities  test,  and  an  information  questionnaire.  After  each  exposure  to  the  VIMT,  which 
lasted  approximately  15  min  (Desktop  group  average  time  of  exposure  was  58  min),  the 
subjects  completed  a  cyber  sickness  checklist  (See  Appendix  C)  to  determine  the 
presence  of  symptoms.  Prior  to  the  retention  test,  the  subjects  completed  a  recall  test, 
requiring  them  to  identify  twenty  various  turret  components  of  the  M1A2  SEP  tank 
depicted  in  photographs  of  the  turret.  Following  the  retention  test,  the  subjects  fdled  out 
two  different  Presence  Questionnaires.  One  presence  questionnaire  was  the  Witmer  and 
Singer  Ver.  3.0,  the  other  was  the  1994  Slater,  Usoh,  and  Steed.  Each  subject  completed 
the  presence  questionnaires  in  random  order.  See  Appendices  D  and  E. 
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CHAPTER  FOUR:  RESULTS 


The  result  from  analysis  of  the  data  eolleeted  is  that  there  is  no  signifieant 
differenee  in  the  adjusted  retention  test  seores  (Task  performanee  times)  between  the 
virtual  reality  group  and  the  desktop  simulation  group,  t  =  QA5,p=  .328.  The  results  are 
shown  in  Table  4,  Table  5,  and  Table  6. 


Table  4.  Descriptive  Statistics  of  Retention  Test  Elapsed  Times  (in  minutes) 


Group 

n 

M 

Mdn 

Trimmed 

M 

SD 

Desktop 

16 

6.907 

6.255 

6.781 

1.591 

VIMT 

16 

6.778 

7.05 

6.795 

1.485 

Note.  The  Trimmed  Mean  is  calculated  by  removing  the  smallest  and  largest  5%,  and 
calculating  the  mean  of  the  remaining  values. 

The  results  in  Table  4  indicate  a  small  difference  exists  between  the  means  for 
experimental  group  raw  times  taken  from  the  retention  test  (See  Table  G-5  in  Appendix 
G).  Regression  of  the  raw  data  according  to  Equation  2  yielded  the  adjusted  score 
equation,  Equation  3,  below. 

Time  =  6.92  -  0.145SpatiaI  +  6.11\Error  -  0.215ER  -  0.4\6Error  x  VR  (3) 


27 


Table  5  (below)  lists  the  results  from  the  linear  regression  that  produeed  Equation 


3,  the  adjusted  seores  equation. 


Table  5.  Results  of  Linear  Regression  for  Time  to  Complete  Task.  From  Equation  3 
(Adjusted  Seore  Equation) 


Variable 

B 

SEB 

P 

Constant 

6.9175 

0.2288 

.000 

Spatial  Ability 

-0.1452 

0.2379 

.547 

Error 

0.7708 

0.2348 

.003 

VR 

-0.2153 

0.2290 

.355 

Error  X  VR 

-0.4158 

0.2404 

.095 

Analysis  of  Varianee 

Souree 

df 

SS 

MS 

F 

P 

Regression 

4 

27.574 

6.894 

4.27 

.008 

Resid.  Error 

27 

43.594 

1.615 

Total 

31 

71.168 

Note.  SD=\.21\,R 

j  _ 

38.7%,  (adj)  = 

29.7% 

Data  from  the  raw  seores  in  Appendix  G  was  then  proeessed  through  Equation  3 
to  yield  the  adjusted  seore  for  eaeh  subjeet.  The  time  differenees  seen  in  the  raw  data 
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were  still  evident  following  adjustment  of  the  seores  using  Equation  3.  A  one-tailed,  t 
test  of  these  adjusted  seores  (See  Table  6,  below  for  eomplete  data)  found  no  signifieant 
difference  between  the  two  groups  of  subjects  {t  =  0.46, =  .674). 


Table  6.  Upper  Tail  t  Test  Analysis  of  Adjusted  Time  to  Complete  Task 


Group 

n 

M 

SD 

SEM 

Desktop 

16 

6.984 

0.871 

0.22 

VIMT 

16 

6.856 

0.729 

0.18 

Note,  t  =  0.45, /»  =  .328,  df=  30.  Ha  =  gDesktop  -  Pvimt  >  0.  SDpooied=  0.328. 
The  equation  used  to  adjust  the  scores  was  Equation  3,  previous  page. 


The  VIMT  group  did  not  score  higher  than  the  Desktop  group  on  the  Slater,  Usoh, 
and  Steed  (SUS)  Presence  Questionnaire  (PQ)  (Not  significant  att  =  -0.37,  p  =  .356),  but 
did  on  the  Witmer  and  Singer  (WS)  PQ  (Significant  att  =  -2.49,  p  =  .01).  See  Tables  7 
and  8,  below.  The  SUS  PQ  asks  more  questions  about  the  degree  of  departure  from  the 
real  world  environment,  while  the  WS  PQ  queries  the  quality  of  the  user’s  interaction  in 
the  VE.  Judging  from  the  results,  the  subjects  indicate  that  the  VR  trainer  presents  a 
more  realistic  interaction  with  the  environment  than  the  desktop  simulation,  but  that  the 
VR  trainer  does  not  completely  engross  them  in  the  artificial  world. 
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Table  7.  SUS  Presence  Questionnaire  Score  Lower-tail  t  Test  Results 


Group 

n 

M 

SD 

SEM 

Desktop 

13 

3.82 

1.38 

0.38 

VIMT 

15 

3.981 

0.804 

0.21 

Note,  t  =  -0.37, p  =  .356,  df=  26,  Ha  =  laoesktop  -  tiviMT  <  0.  SDpooied=  1  •  1 1  •  The 
higher  the  score,  the  higher  the  sense  of  presence  on  a  scale  of  1  to  7. 


Table  8.  WS  Presence  Questionnaire  Lower- tail  t  Test  Results 


Group 

n 

M 

SD 

SEM 

Desktop 

13 

4.45 

1.21 

0.34 

VIMT 

16 

5.529 

0.928 

0.23 

Note,  t  =  -2.49, = 

.010,  J/=  27,  Ha  = 

-  gDesktop  -  gVIMT  <  0.  SD  pooled-  1-06.  Thc 

higher  the  score,  the  higher  the  sense 

of  presence  on  a  scale  of  1  to  7. 

The  experimental  group  averages  from  the  WS  Presence  Questionnaire,  The  SUS 
Presence  Questionnaire,  and  the  groups’  raw  retention  test  performance  times  were  tested 
for  correlation  and  are  presented  in  Table  9,  below.  There  is  no  significant  correlation 
between  the  VR  group’s  performance  times  and  the  PQs,  indicating  no  relationship  exists 
between  the  performance  times  and  the  PQ  scores.  The  correlation  between  the  desktop 
group’s  performance  times  and  their  PQ  scores  is  slightly  stronger,  but  still  insignificant. 
There  is  a  slightly  larger  correlation  between  the  times  of  both  groups  and  the  WS  PQ, 
than  with  the  SUS  PQ. 
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Table  9.  Correlation  between  Raw  Retention  Test  Performance  Times  and  PQ  Scores. 


Group 

SUS  PQ 

WS  PQ 

Desktop 

Correlation 

-.201 

-.441 

p  value 

.510 

.131 

VIMT 

Correlation 

-.118 

-.186 

p  value 

.675 

.490 

Note.  Although  insignificant  in  this  case,  negative  correlation  would  indicate  that 
Higher  PQ  scores  may  be  related  to  faster  performance  times. 


The  correlation  between  the  training  time  improvement  between  the  first  and 
second  training  trials  is  displayed  in  Table  10,  below.  There  is  a  strong  correlation 
between  the  individual  improvement  for  the  desktop  group  and  the  individual  scores  from 
both  PQs,  indicating  a  possible  positive  relationship  between  the  two.  That  positive 
relationship  would  mean  that  higher  PQ  scores  would  relate  to  larger  improvements  in 
performance  times.  The  correlation  between  the  VR  group  improvement  times  and  their 
respective  PQ  scores  is  weak.  However,  training  trial  times  of  the  desktop  group  were  on 
average  58  minutes,  while  the  VR  group  averaged  about  15  min.  Their  respective 
improvements  were  17.1  min  and  3.63  min.  The  desktop  group  thus  had  a  lot  of  room  to 
improve — this  large  improvement  no  doubt  positively  affected  the  correlation  of  the 
desktop  group. 
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Table  10.  Correlation  between  Training  Time  Improvement  and  PQ  Seores. 


Group 

SUS  PQ 

WS  PQ 

Desktop 

Correlation 

.854 

.929 

p  value 

.030 

.002 

VIMT 

Correlation 

-.236 

.270 

p  value 

.397 

.311 

Note.  Positive  eorrelation  between  both  PQs  and  the  Desktop  group  improvement  in 
training  performanee  seores  indieates  that  inereased  PQ  seores  appear  to  be  related  to 
inereased  improvement  of  training  performanee  times. 


The  results  from  the  20  question  reeall  test  (See  Table  11,  below)  administered 
prior  to  the  retention  test  indieate  that  there  is  not  a  signifieant  differenee  between  the 
seores  of  the  desktop  group  and  the  VR  group  {t  =  0.79, />  =  .222).  The  reeall  test  figures 
appear  in  Appendix  H. 
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Table  11.  Recall  Test  t  Test  Results 


Group 

n 

M 

SD 

SEM 

Desktop 

10 

17.5 

1.51 

0.48 

VIMT 

10 

16.7 

2.83 

0.90 

Note,  t  0.75,  p  .222,  df  13,  Ha  ^Desktop  "  ^VIMT  ^  0.  SDpooled  1.06 


Of  the  16  VR  subjects,  9  reported  symptoms  of  cyber  sickness  during  training 
with  the  VIMT.  None  of  the  participants  felt  they  needed  to  cease  training  due  to 
discomfort,  nor  complained  of  symptoms  during  training.  A  numerical  tally  of  reported 
symptoms  and  their  seriousness  is  below  in  Table  12. 
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Table  12.  Tallied  Number  and  Severity  of  VR  Group  Cyber  Siekness  Symptoms 

Symptom  Severity 

Slight  Moderate  Severe 


General  Diseomfort  2 

Fatigue  1 

Headaehe/Neek  Strain  3 

Eye  Strain  5 

Diffieulty  Foeusing  2 

Inereased  Salivation 
Sweating  1 

Nausea  1 


Diffieulty  Coneentrating  2 

Fullness  of  Head  4 

Blurred  Vision  2 

Dizzy  (Eyes  Open) 

Dizzy  (Eyes  Closed)  1 

Vertigo 


1 

2 


Note,  n  =  \6.  Symptoms  taken  from  Cyber  Siekness  Cheeklist,  Appendix  C. 
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The  number  of  proeedural  errors  made  by  the  VR  group  during  the  retention  test 


was  not  signifieantly  different  (See  Table  13,  below)  from  the  number  of  proeedural 
errors  made  by  the  desktop  group  {t  =  1.02, =  .315). 


Table  13.  One-tailed  t  Test  Results  of  Retention  Test  Proeedural  Errors 


Group 

n 

M 

SD 

SEM 

Desktop 

16 

1.38 

1.45 

0.36 

VIMT 

16 

1.88 

1.31 

0.33 

Note,  t  =  \  .02,p  = 

.315,  J/=30,  Ha 

~  P  Desktop  ■ 

^VIMT  0.  SD  pooled 

1.38 

The  interaetion  of  proeedural  errors  and  the  VR  qualitative  variable  yielded  a 
relatively  high p  value  in  the  regression  of  Equation  2  (p  =  .095).  The  interaetion  term’s 
eoeffieient  also  has  the  seeond  highest  value  in  the  regressed  equation  {B4  =  -0.416). 
Graphing  the  interaetion  (See  Eigure  3,  below),  the  mean  performanee  times  of  VR  group 
had  a  smaller  varianee  than  did  the  mean  times  of  their  desktop-trained  eounterparts. 
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2  =  VR  Simulation 


Error  Level 

■  2  (High) 
•  1  (Low) 


Ligure  3.  Interaction  of  Error  and  VR  Variables  with  Respect  to  Performance  Time 
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CHAPTER  FIVE:  DISCUSSION 


The  results  from  the  statistieal  analysis  do  not  signifioantly  support  the  hypothesis 
that  VR  inereases  retention  of  the  trained  task  in  this  case.  A  small  difference  was  found 
between  the  times  to  perform  the  transfer  task  for  the  two  experimental  groups. 

The  procedural  errors  made  by  the  two  groups  of  subjects  during  the  retention  test 
are  not  significantly  different,  but  the  VR  group  made  slightly  more  errors.  These  errors 
may  have  had  a  detrimental  effect  on  the  VR  group’s  performance  time  during  the  test. 

The  interaction  of  the  VR  variable  with  the  number  of  retention  test  errors  showed 
that  the  VR  group  had  less  variance  in  their  retention  test  performance  times  than  did  the 
desktop  group.  A  possible  explanation  of  this  result  may  be  the  increased  specificity  of 
location  cues  in  the  VR  model.  The  VR  subjects  were  required  to  reach  out  to  the  actual 
switch  or  lever  location  to  operate  the  selected  mechanism,  while  the  desktop  group 
maneuvered  a  mouse  on  a  fiat  desk  to  perform  the  same  function.  That  increased  realism 
of  the  VR  experience  may  aid  in  retrieval  of  location  cues  from  memory,  and  be  the 
cause  of  the  decreased  variance  in  the  VR  group  performance  times.  This  increased 
consistency  in  performance  may  be  an  added  benefit  of  VR  training.  However,  in  this 
case,  the  difference  in  the  overall  performance  times  between  the  two  groups,  and  their 


37 


relation  to  the  ACT-R  model  (Anderson  &  Lebiere,  1998)  was  still  the  overarehing 
eoneern  of  the  study. 

The  VR  group  reeorded  signifioantly  higher  Witmer  and  Singer  PQ  seores 
{t  =  -2.49,  p  =  .010)  than  did  the  desktop  group.  The  differenees  between  the  two 
experimental  groups  on  the  Slater,  Usoh,  and  Steed  PQs  was  not  signifioant 
{t  =  -0.37,  p  =  .356).  The  higher  Witmer  and  Singer  PQ  seores  of  the  VR  group  indieate 
that  the  group  experienced  higher  levels  of  presence  than  did  those  using  the  desktop 
simulation.  The  difference  in  presence  between  the  experimental  groups  did  not  seem  to 
have  a  significant  effect  on  their  retention  test  time  and  errors,  however.  The  WS  PQ 
asks  more  questions  as  to  the  quality  of  the  interaction  with  the  VE,  while  the  SUS  PQ 
asks  more  questions  as  to  the  user’s  feeling  of  departure  from  the  real  world,  and 
inclusion  in  the  VE.  Therefore,  while  the  quality  of  the  interaction  was  better  in  the  VR 
than  in  the  desktop  simulation,  the  subjects  did  not  significantly  feel  more  excluded  from 
the  real  world  and  immersed  in  the  artificial  environment  than  did  the  group  using  the 
desktop  simulation. 

The  subjects  that  participated  in  the  experiment  are  not  novices  on  the  Ml  series 
tank.  Although  at  the  time  of  the  experiment,  they  were  new  to  the  specific  model  of  the 
vehicle,  the  Ml  A2  SEP,  the  subjects  had  been  training  on  the  earlier  models  for  over  4 
months  and  were  well  acquainted  with  the  layout  of  the  systems  and  panels  in  the  turret. 
Since  a  good  portion  of  the  advantages  of  VR  is  in  spatial  knowledge,  the  differences 
between  the  experimental  groups  may  not  be  so  evident  when  both  groups  have  prior 
knowledge  of  the  environment.  This  factor  of  prior  experience  may  be  one  of  the  reasons 
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why  the  differences  in  presence  scores  between  the  two  groups  did  not  manifest  itself  in 
the  groups’  adjusted  scores.  The  activation  of  the  memory  elements  in  the  VR  group 
versus  the  desktop  group  may  have  been  different,  but  the  interference  generated  by  prior 
training  may  have  masked  the  difference,  resulting  in  statistically  similar  scores.  A  study 
using  subjects  with  no  prior  knowledge  of  the  Ml  tank  may  yield  differences  that  are 
more  significant. 

However,  despite  their  prior  experience,  there  is  a  slight  advantage  for  the  VR 
trained  group  in  performance  times  during  the  retention  test. 

From  the  standpoint  of  experienced  trainees,  it  is  evident  that  a  VR  trainer  would 
have  a  greater  effect  if  used  earlier  in  the  training  evolution.  As  with  the  VR  Ml  Tank 
Driver  Trainer  currently  in  use  to  train  tank  drivers,  the  students  drive  in  the  VR  trainer 
prior  to  operating  the  real  world  vehicle.  The  driver  trainer  would  likely  have  a  smaller 
impact  on  a  trained  tank  driver.  Since  there  is  currently  no  plan  to  change  the  training 
evolution  of  the  Ml  tank  mechanic  program,  where  mechanics  progress  from  the  oldest 
model  of  the  tank  to  the  newest  model,  there  appears  to  be  less  value  in  using  a  VR 
maintenance  trainer  late  in  the  curriculum.  With  the  current  training  program,  the 
subjects  used  in  this  experiment  were  taken  from  the  target  population  of  mechanics,  and 
a  fielded  system  placed  into  this  program  would  likely  produce  performance  results 
similar  to  those  found  in  this  experiment. 

The  maintenance  task  selected  for  this  study  is  not  the  type  of  task  normally 
trained  in  VR.  Navigational  tasks  are  typically  selected  for  training  in  virtual  reality 
because  such  tasks  require  a  large  degree  of  spatial  awareness,  something  VR  presents 
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with  realistic  accuracy.  The  Weapon  System  InterFIT  requires  some  spatial  knowledge 
of  the  interior  layout  of  the  M1A2  SEP  Tank  turret,  but  mostly  the  knowledge  required 
by  the  task  is  procedural  in  nature.  Although  procedural  tasks  can  be  trained  in  VR, 
experience  has  shown  that  performance  of  such  tasks  in  VR  may  not  significantly  differ 
than  that  in  less  expensive  simulation  environments,  such  as  desktop  simulation.  See 
Table  14  for  a  comparison  of  tasks  and  their  respective  demands. 

There  are  cost  considerations  when  fielding  a  VR  training  system  versus  a 
desktop  training  system.  The  addition  of  an  HMD  and  tracking  system  to  a  desktop 
trainer  increases  the  unit  cost  more  than  $10  thousand.  This  does  not  include  the 
additional  cost  of  any  proprietary  software  that  must  be  purchased  or  developed  for  the 
VR  equipment  to  operate  properly.  The  desktop  simulation  obviously  has  the  advantage 
of  relying  on  simple,  relatively  inexpensive  technology.  Given  no  significant  difference 
in  the  performance  of  a  VR  simulation  and  a  desktop  simulation,  the  desktop  simulation 
would  be  the  better  choice  from  a  purely  fiscal  standpoint. 
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Table  14.  Appropriateness  of  Training  Tasks  for  Training  in  Virtual  Reality 


Task  Types 

Primary  Task  Demands 

Appropriateness 

for  VE  Training 

Rationale 

Navigation 

Spatial  Knowledge  of 

Environment 

High 

Spatial  Knowledge 

main  added  value 

ofVR 

Forestry 

Operations 

Eye-hand  coordination  / 

Use  of  Controls 

Medium 

Eess  use  of  spatial 

knowledge. 

Natural  interface 

assists  in  task 

Maintenance 

Procedural  Knowledge 

Eow 

Procedural 

of  T ask  knowledge  more 

easily  replieable  by 
other  means 

Cyber  siekness  was  not  a  faetor  in  the  exeeution  of  the  experiment.  Although  9  of 
the  16  VR  reported  “slight”  to  “moderate”  symptoms  of  eyber  sickness  following 
exposure  (See  Table  12,  Chapter  3)  none  felt  that  they  could  not  continue  with  the 
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training.  The  HMD  used  in  the  study  eould  be  adjusted  for  IPD  and  viewing  angle.  It  did 
not  have  a  diopter  adjustment,  but  did  aeeommodate  the  wear  of  eyeglasses.  This  laek  of 
foeusing  adjustment  likely  ereated  some  eyestrain  among  the  subjects  over  time.  The 
HMD  used  did  have  sufficient  resolution  to  see  all  of  the  text  displayed  inside  the  tank. 
One  subject  initially  reported  the  inability  to  read  the  labeling  inside  the  environment,  but 
when  the  subject  wore  his  glasses  to  correct  his  astigmatism,  he  reported  no  vision 
problems.  If  the  VIMT  were  fielded  to  replace  desktop  systems,  it  would  be 
recommended  that  the  HMD  used  in  the  permanent  system  have  adjustments  for  diopter 
power,  as  well  as  IPD  and  viewing  angle  (Menozzi,  2000).  In  addition,  the  fielded 
system  should  have  the  ability  to  be  used  without  a  HMD,  so  that  an  instructor  or  subject 
not  able  to  wear  HMDs  can  operate  the  system  without  difficulty.  A  controller  such  as  a 
Logitech®  Spaceball®  would  work  quite  well  as  a  backup  controller  to  navigate  though 
the  3D  space. 

The  experiment  performed  may  have  been  enhanced  by  a  larger  number  of 
training  sessions,  and  by  a  longer  delay  period  between  training  and  the  retention  test. 

The  larger  number  of  training  sessions  would  allow  increased  activation  of  the  memory 
elements  associated  with  the  training  task,  while  the  increased  delay  period  would 
increase  the  amount  of  decay  present  at  the  time  of  the  retention  test.  These  two 
improvements  may  show  a  significant  difference  in  performance  scores  between  those 
trained  in  VR  and  those  trained  on  desktop  simulators.  Additionally,  as  mentioned 
earlier,  use  of  inexperienced  subjects  may  yield  different  findings  than  those  found  using 
subjects  with  prior  training  on  older  models  of  the  systems. 
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The  results  of  the  experiment  determined  that  the  training  eondueted  on  both  the 
VR  and  desktop  systems  positively  transferred  to  the  real  world  system.  Additionally, 
the  VR  system  showed  that  it  deereased  the  varianee  of  performanee  times  of  the  subjeets 
that  used  it.  However,  given  the  eonsiderable  eost  differential  between  the  VR  and 
desktop  systems,  as  well  as  the  eurrent  evolution  of  meehanie  training  in  the  U.S.  Army, 
the  desktop  trainer  seems  to  provide  better  eost  benefits. 

Although  this  study  did  not  find  a  signifieant  differenee  in  the  retention  between 
those  trained  in  VR  environments,  and  those  trained  in  desktop  simulation  environments, 
it  explored  a  previously  unstudied  effeet  of  virtual  reality  on  humans.  If,  after  further 
studies,  it  were  found  that  virtual  reality  inereases  retention  of  the  trained  tasks,  it  would 
likely  eause  a  shift  in  training  strategies  in  several  industries.  Beeause  of  its  potential 
importanee  to  the  training  of  personnel,  more  effort  should  be  applied  in  the  future 
towards  determining  the  aetual  effeets  of  VR  on  memory. 
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APPENDIX  A 


Experiment  Pre-Brief 
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Experiment  Pre-Brief 


You  have  been  seleeted  as  a  researeh  partieipant  to  determine  if  using  a  virtual 
reality  simulator  versus  a  desktop  simulator  inereases  retention  of  the  trained  task.  Some 
of  you  will  be  immersed  in  a  virtual  environment  (Ml  A2  SEP  turret)  and  perform  system 
power-up,  and  the  Weapon  System,  GPS,  Loader’s  Station  InterFIT.  Along  with  regular 
training,  you  will  reeeive  a  pre-test  and  post-test  of  the  above  tasks,  and  be  tested  on  the 
two  tasks  onee  per  day  for  four  days  starting  today,  to  assess  your  skill  development.  The 
elapsed  time  and  the  errors  per  task  will  be  measured.  Two  days  following  your  post¬ 
test,  you  will  again  be  tested  on  the  two  above  tasks,  but  in  the  aetual  tank,  or  in  the 
Hands-on  Training  System  (HOTS). 

You  will  also  be  given  several  surveys  to  eomplete. 

Beeause  some  of  you  will  be  viewing  the  virtual  environment  (VE)  through  a 
helmet-mounted  display  (HMD),  you  may  experienee  motion  siekness  or  eyestrain.  If 
your  symptoms  beeome  too  extreme,  stop  work  and  notify  an  instruetor  or  an 
experimenter. 

If  you  have  questions,  do  not  hesitate  to  ask  an  experimenter. 
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APPENDIX  B 


Maintenance  Task  Performance  Measures 
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Table  B-1,  System  Power-up  Task  Proeedures 


Step 

Proeedure  Steps 

Action 

Results 

1. 

Tank  parked,  preferable 
on  hard  level  surfaee 

No  action  required. 

Tank  assumed  to  be 
on  a  hard  level 
surface. 

2. 

Traeks  bloeked 

No  action  required 

Assumed  that  tracks 
are  blocked 

3. 

Parking  brake  set 

Click  on  the  parking  brake 
to  set,  if  required. 

Parking  brake  is  set. 
Warning  message  is 
displayed. 

4. 

Batteries  servieed 

No  action  required 

Assume  batteries  are 
serviced. 

5. 

Turret  power  on 

Press  the  Master  power 
and  Turret  power 
pushbuttons,  or  press 
turret  power  pushbutton. 

Turret  power  is 
turned  on. 

6. 

All  eireuit  breakers  on  or 
in  tripped  eondition 

No  action  required,  unless 
CB  were  turned  off  or 
tripped  during 
verification. 

7. 

Commander's  Independent 
Thermal  Viewer  (CITY) 
on 

Press  the  CITY  power 
pushbutton. 

Powers  up  defaults  to 
Normal,  light  illuminates 
at  18  seconds 

CITY  comes  out  of 
the  stowed  position. 

8. 

FIRE  CONTROL  MODE 
switeh  on  Gunner's 

Primary  Sight  (GPS)  set  to 
NORMAL 

Click  on  the  FIRE 
CONTROL  MODE 
switch  and  set  to  the 
NORMAL  position 

FIRE  CONTROL 
MODE  switch  is 
placed  to  the 
NORMAL  position, 
and  light  illuminates. 

9. 

Gun  Turret  Drive  (GTD) 
switeh  on  Loader's  Panel 
(LP)  set  to  POWERED 

Click  on  the  Gun  Turret 
Drive  switch  and  set  to  the 
POWERED  position 

Gun  Turret  Drive 
switch  is  placed  to 
the  POWERED 
position,  and  light 
illuminates. 
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Step 

Procedure  Steps 

Action 

Results 

10. 

Gun  and  turret  locks 
locked  unless  directed 
otherwise  by  procedure 

Click  on  the  gun  travel 
lock  and  set  to  the  locked 
position.  Click  on  the 
turret  traverse  lock  and  set 
to  the  locked  position. 

Gun  and  turret  locks 
are  locked. 

11. 

Main  gun  positioned  over 
front  of  vehicle 

Using  the  CCHA,  GCH, 
Manual  elevation,  place 
gun  over  the  front  of  the 
vehicle. 

Main  gun  is  placed 
over  the  front  of  the 
vehicle. 

(Over  driver’s  hatch) 

12. 

Communication  system 
set  up  for  normal 
operation 

Not  required 
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Table  B-2.  Weapon  System  InterFIT  Task  Proeedures 


Step 

Proeedure  Steps 

Action 

Results 

13. 

Press  the  START  TEST 

Initial  page  of  GPS 

menu  button  on  the 

Loader’s  station 

GCDP. 

InterFIT  is 

displayed. 

14. 

1 .  Click  on  the  MAIN 

MAIN  GUN 

GUN  SAFE/ARMED 

SAFE/ARMED 

SWITCH  and  set  to 

SWITCH  is  set  to 

the  SAFE  position. 

the  SAFE  position. 

2.  Click  on  the 

GUN/TURRET 

GUN/TURRET 

DRIVE  SWITCH  is 

DRIVE  SWITCH  and 

set  to  MANUAL. 

set  to  MANUAL. 

3.  Click  on  the  TURRET 

TURRET  BLOWER 

BLOWER  SWITCH 

SWITCH  is  set  to 

and  set  to  OFF. 

OFF. 

4.  Click  on  the  FIRE 

FIRE  CONTROL 

CONTROL  MODE 

MODE  SWITCH  is 

SWITCH  and  set  to 

set  to  NORMAL. 

NORMAL. 

5.  Click  on  the  GUN 

GUN  SELECT 

SELECT  TRIGGER 

TRIGGER  is  set  to 

and  set  to  TRIGGER 

TRIGGER  SAFE. 

SAFE. 

6.  Click  on  the 

DEFROSTER 

DEFROSTER 

SWITCH  and  set  to 

SWITCH  and  set  to 

OFF. 

OFF. 

7.  SELECT  DONE 

Next  menu  is 

MENU  KEY. 

displayed 
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Step 

Procedure  Steps 

Action 

Results 

15. 

1. 

Click  on  the  TURRET 

TURRET  BLOWER 

BLOWER  SWITCH 

SWITCH  is  set  to 

to  set  to  ON. 

ON. 

2. 

Click  on  the 

GUN/TURRET 

GUN/TURRET 

DRIVE  is  set  to 

DRIVE  to  set  to 
POWERED. 

POWERED. 

3. 

Click  on  the  GPS 

GPS  DEEROSTER 

DEEROSTER 

SWITCH  is  set  to 

SWITCH  to  set  to  ON. 

ON. 

4. 

Click  on  the  MAIN 

MAIN  GUN 

GUN  SAEE/ARMED 

SAEE/ARMED 

LEVER  to  set  to 

LEVER  is  set  to 

ARM. 

ARM. 

5. 

SELECT  MENU 

Next  menu  is 

DONE  KEY 

displayed. 

16. 

Press  YES  or  NO  for 
defroster  light  status. 

17. 

1. 

Click  on  the  TURRET 

TURRET  BLOWER 

BLOWER  SWITCH 

SWITCH  is  set  to 

to  set  to  OEE 

OEE. 

2. 

Click  on  the 

GUN/TURRET 

GUN/TURRET 

DRIVE  SWITCH  is 

DRIVE  SWITCH  to 
set  to  EL  UNCPL 

set  to  EL  UNCPL. 

3. 

Click  on  the  GPS 

GPS  DEEROSTER 

DEEROSTER 

SWITCH  is  set  to 

SWITCH  to  set  to 

OEE 

OEE 

4. 

Click  on  the  MAIN 

MAIN  GUN. 

GUN  SAEE/ARMED 

SAEE/ARMED 

LEVER  to  set  to 

LEVER  is  set  to 

SALE 

SALE. 

SELECT  DONE  MENU 

Next  menu  is 

KEY 

displayed. 
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Step 

Procedure  Steps 

Action 

Results 

18. 

1 .  Click  on  the  SABOT 
SWITCH. 

SABOT  EIGHT  is 
illuminated. 

2.  Click  on  the  HEAT 
SWITCH. 

HEAT  is 
illuminated. 

3.  Click  on  the 
AIR/GROUND 
SWITCH 

AIR/GROUND 
EIGHT  is 
illuminated 

4.  Click  on  the  STAFF 
SWITCH. 

STAFF  EIGHT  is 
illuminated 

5.  Click  on  the  MPAT 
SWITCH. 

MPAT  EIGHT  is 
illuminated. 

6.  Click  on  the  FIGHTS 
TEST  SWITCH. 

All  lights  on  the 
Eoader’s  Panel  and 
GPS  are  illuminated. 

Press  DONE  menu  key. 

Next  menu  is 
displayed. 

19. 

Click  on  the  GUN 

SEFECT  SWITCH  and 
set  to  MAIN  and  then 
release. 

GUN  SEFECT 
SWITCH  is  set  to 
MAIN.  MAIN 

GUN  SEFECT  lamp 
illuminates. 

Press  DONE  menu  key. 

Next  menu  is 
displayed. 

20. 

Click  on  the  GUN 

SEFECT  SWITCH  and 
set  to  COAX  and  then 
release. 

GUN  SEFECT 
SWITCH  is  set  to 
COAX. 

COAX  GUN 

SEFECT  lamp 
illuminates. 

Press  DONE  menu  key. 

Next  menu  is 
displayed. 
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Step 

Procedure  Steps 

Action 

Results 

21. 

Click  on  the  FIRE 
CONTROL  MODE  and 
set  to  EMERGENCY  and 
then  release. 

Press  DONE  menu  key. 

FIRE  CONTROL 
MODE  SWITCH  is 

set  to 

EMERGENCY. 

FIRE  CONTROL 
MODE 

EMERGENCY  lamp 
illuminates. 

Next  menu  is 
displayed 

22. 

Click  on  the  FIRE 
CONTROL  MODE  and 
set  to  MANUAL  and  then 
release. 

Press  DONE  menu  key. 

FIRE  CONTROL 
MODE  SWITCH  is 
set  to  MANUAL. 
FIRE  CONTROL 
MODE  MANUAL 
lamp  illuminates. 

Next  menu  is 
displayed 

23. 

Click  an  drag  the  PANEL 
LIGHTS  KNOB  fully 
clockwise 

SELECT  DONE  MENU 
KEY 

All  lights  are 
illuminated. 

24. 

ARE  ANY  FC  MODE  OR 

Press  YES  to  answer 

GUN  SELECT  MODE 
LAMPS  LIT? 

question 

Next  menu  is 
displayed 

25. 

ARE  ANY  LAMPS  ON 

Press  YES  to  answer 

THE  LOADER  PANEL 
LIT? 

question 

Next  menu  is 
displayed 
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Step 

Procedure  Steps 

Action 

Results 

26. 

LAMP  CHECK 

MAIN  GUN  ARMED 
MAIN  GUN  SAEE 
GUN/TURRET  DRIVE 

EE  UNCPL 

GUN/TURRET  DRIVE 
POWERED 

GUN/TURRET  DRIVE 
MANUAE 

Press  YES  for  each  lamp, 
and  then  press  DONE 

Next  menu  is 
displayed 

27. 

LAMP  CHECK 

SABOT 

MPAT 

AIR 

GROUND 

STAEE 

HEAT 

Press  YES  for  each  lamp, 
and  then  press  DONE 

Next  menu  is 
displayed 
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APPENDIX  C 


Cyber  Sickness  Symptom  Checklist 
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Cyber  Sickness  Symptom  Checklist 


Format  and  some  content  checklist  from  Bums  &  Patrey  (2000). 

Participant  Name _  Date _ 

Before  Training _  After  Training _  (Check  only  ONE) 

Duration  spent  using  head-mounted  visual  display  today  (in  minutes): _ 

Instmctions:  Please  circle  the  severity  of  any  symptoms  that  apply  to  you  right  now. 


1. 

General  Discomfort 

None 

Slight 

Moderate 

Severe 

2. 

Fatigue 

None 

Slight 

Moderate 

Severe 

3. 

Headache/Neck  Strain 

None 

Slight 

Moderate 

Severe 

4. 

Eye  Strain 

None 

Slight 

Moderate 

Severe 

5. 

Difficulty  Focusing 

None 

Slight 

Moderate 

Severe 

6. 

Increased  Salivation 

None 

Slight 

Moderate 

Severe 

7. 

Sweating 

None 

Slight 

Moderate 

Severe 

8. 

Nausea 

None 

Slight 

Moderate 

Severe 

9. 

Difficulty  Concentrating 

None 

Slight 

Moderate 

Severe 

10. 

Fullness  of  Head* 

None 

Slight 

Moderate 

Severe 

*Fullness  of  head  means  internal  pressure  in  head,  similar  to  sinus  pressure,  such  as  one 
gets  when  hanging  upside  down  from  the  monkey  bars 


11. 

Blurred  Vision 

None 

Slight 

Moderate 

Severe 

12. 

Dizzy  (Eyes  Open) 

None 

Slight 

Moderate 

Severe 

13. 

Dizzy  (Eyes  Closed) 

None 

Slight 

Moderate 

Severe 

14. 

** 

Vertigo 

None 

Slight 

Moderate 

Severe 

Vertigo  is  a  disordered  state  in  which  the  person  or  his/her  surroundings  seem  to  whirl 
dizzily;  loss  of  orientation  that  makes  it  difficult  to  perceive  which  way  is  up 

Are  there  any  other  symptoms  that  you  are  experiencing  right  now?  If  so,  please  describe  the 
symptom(s)  and  rate  their  severity  on  the  other  side. 
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APPENDIX  D 


Witmer  and  Singer  Presence  Questionnaire 
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WS  Presence  Questionnaire 


(Witmer  &  Singer,  Ver.  3.0,  Nov.  1994) 


Soldier  Name: _ 

Characterize  your  experience  in  the  environment,  by  marking  an  "X"  in  the  appropriate 
box  of  the  7-point  scale,  in  accordance  with  the  question  content  and  descriptive  labels.  Please 
consider  the  entire  scale  when  making  your  responses,  as  the  intermediate  levels  may  apply. 
Answer  the  questions  independently  in  the  order  that  they  appear.  Do  not  skip  questions  or  return 
to  a  previous  question  to  change  your  answer. 

WITH  REGARD  TO  THE  EXPERIENCED  ENVIRONMENT 

1.  How  much  were  you  able  to  control  events? 


NOT  AT  ALE  SOMEWHAT  COMPLETELY 


2.  How  responsive  was  the  environment  to  actions  that  you  initiated  (or 
performed)? 


NOT  MODERATELY  COMPLETELY 

RESPONSIVE  RESPONSIVE  RESPONSIVE 


3.  How  natural  did  your  interactions  with  the  environment  seem? 


EXTREMELY  BORDERLINE  COMPLETELY 

ARTIEICIAL  NATURAL 


4.  How  much  did  the  visual  aspects  of  the  environment  involve  you? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 
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5.  How  much  did  the  auditory  aspects  of  the  environment  involve  you? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


6.  How  natural  was  the  meehanism  that  eontrolled  movement  through  the 
environment? 


EXTREMEEY 

BORDERLINE 

COMPLETELY 

ARTIEICIAL 

NATURAL 

NATURAL 

7.  How  compelling  was  your  sense  of  objeets  moving  through  space? 


NOT  AT  ALL 

MODERATEEY 

VERY 

COMPELLING 

COMPEEEING 

8.  How  mueh  did  your  experienees  in  the  virtual  environment  seem  eonsistent 

your  real  world  experienees? 

1  1  1 

1  1 

NOT 

MODERATEEY 

VERY 

CONSISTENT 

CONSISTENT 

CONSISTENT 

9.  Were  you  able  to  antieipate  what  would  happen  next  in  response  to  the  aetions 
that  you  performed? 


NOT  AT  ALL  SOMEWHAT  COMPEETEEY 


10.  How  completely  were  you  able  to  aetively  survey  or  seareh  the  environment 
using  vision? 


NOT  AT  AEE  SOMEWHAT  COMPEETEEY 
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1 1 .  How  well  could  you  identify  sounds? 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


12.  How  well  could  you  localize  sounds? 


NOT  AT  ALL  SOMEWHAT  COMPEETEEY 


13.  How  well  could  you  actively  survey  or  search  the  virtual  environment  using 

touch? 


NOT  AT  AEE  SOMEWHAT  COMPEETEEY 


14.  How  compelling  was  your  sense  of  moving  around  inside  the  virtual 
environment? 


NOT  MODERATEEY  VERY 

COMPEEEING  COMPEEEING  COMPEEEING 


15.  How  closely  were  you  able  to  examine  objects? 


NOT  AT  ALE 


PRETTY 

CEOSEEY 


VERY 

CEOSEEY 


16.  How  well  could  you  examine  objects  from  multiple  viewpoints? 


NOT  AT  ALE 


SOMEWHAT 


EXTENSIVEEY 
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17.  How  well  could  you  move  or  manipulate  objects  in  the  virtual  environment? 


NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 


18.  How  involved  were  you  in  the  virtual  environment  experience? 


NOT  MILDLY  COMPLETELY 

INVOLVED  INVOEVED  ENGROSSED 


19.  How  much  delay  did  you  experience  between  your  actions  and  expected 
outcomes? 


NO  DELAYS  MODERATE  EONG 

DEEAYS  DEEAYS 


20.  How  quickly  did  you  adjust  to  the  virtual  environment  experience? 


NOT  AT  AEE 

SEOWLY 

EESS  THAN 

ONE  MINUTE 

2 1 .  How  proficient 

in  moving  and  interacting  with  the  virtual  environment  did 

you  feel  at  the  end  of  the  experience? 

1  1  1 

NOT 

REASONABEY 

VERY 

PROEICIENT 

PROEICIENT 

PROEICIENT 

22.  How  much  did  the  visual  display  quality  interfere  or  distract  you  from 

performing  assigned  tasks  or  required  activities? 

1  1  1 

NOT  AT  AEE 

INTEREERED 

PREVENTED 

SOMEWHAT 

TASK  PEREORMANCE 
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23.  How  much  did  the  control  devices  interfere  with  the  performance  of  assigned 
tasks  or  with  other  activities? 


NOT  AT  ALL 


INTERFERED 

SOMEWHAT 


INTERFERED 

GREATEY 


24.  How  well  could  you  concentrate  on  the  assigned  tasks  or  required  activities 
rather  than  on  the  mechanisms  used  to  perform  those  tasks  or  activities? 


NOT  AT  ALE  SOMEWHAT  COMPEETEEY 


25.  How  completely  were  your  senses  engaged  in  this  experience? 


NOT  MILDEY  COMPLETEEY 

ENGAGED  ENGAGED  ENGAGED 


26.  To  what  extent  did  events  occurring  outside  the  virtual  environment  distract 
from  your  experience  in  the  virtual  environment? 


NOT  AT  ALE 


MODERATEEY 


VERY  MUCH 


27.  Overall,  how  much  did  you  focus  on  using  the  display  and  control  devices 
instead  of  the  virtual  experience  and  experimental  tasks? 


NOT  AT  AEE  SOMEWHAT  VERY  MUCH 


28.  Were  you  involved  in  the  experimental  task  to  the  extent  that  you  lost  track  of 

time? 


NOT  AT  AEE 


SOMEWHAT 


COMPEETEEY 
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29.  How  easy  was  it  to  identify  objeets  through  physieal  interaetion;  like 
touehing  an  objeet,  walking  over  a  surfaee,  or  bumping  into  a  wall  or  objeet? 


IMPOSSIBLE  MODERATELY  VERY  EASY 

DIEEICULT 

30.  Were  there  moments  during  the  virtual  environment  experienee  when  you  felt 
eompletely  foeused  on  the  task  or  environment? 


NONE  OCCASIONALLY  EREQUENTLY 


3 1 .  How  easily  did  you  adjust  to  the  eontrol  deviees  used  to  interaet  with  the 
virtual  environment? 


DIEEICULT 


MODERATE 


EASILY 


32.  Was  the  information  provided  through  different  senses  in  the  virtual 
environment  (e.g.,  vision,  hearing,  toueh)  eonsistent? 


NOT 

CONSISTENT 


SOMEWHAT 

CONSISTENT 


VERY 

CONSISTENT 
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APPENDIX  E 


Slater,  Usoh,  Steed  Presence  Questionnaire 
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SUS  Presence  Questionnaire 


(Slater,  Usoh,  Steed,  1994) 

Part  A 

A2;  The  following  deseribes  my  educational  level: 


Educational  Level 

Please 

tick 

against 

your 

answer 

1. 

GED 

2. 

High  School  Graduate 

3. 

Some  College 

4. 

Associate ’s  Degree 

5. 

Bachelor ’s  Degree 

6. 

Graduate  Student 

7. 

Master ’s  Degree 

A3.  Have  you  experienced  "virtual  reality"  (Ml  driver’s  trainer,  or  other 
immersive  environments)  before? 


/  have  experience  virtual  reality 

Please 

tick 

against 

your 

answer 

1 .  never  before 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  a  great  deal 

7 
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A4.  To  what  extent  do  you  use  a  computer  in  your  daily  activities? 


I  use  a  computer... 

Please  tick 

against 

your 

answer 

1 .  not  at  all 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  very  much  so 

7 

A5.  Think  about  some  other  place  you’ve  been  today  (e.g.,  your  home,  barracks 
room,  car,  or  any  other  place). 

What  is  the  place  you  are  thinking  of? . 

A6.  The  following  questions  relate  to  the  characteristics  of  how  you  are  thinking 
about  that  place. 

(a)  In  you’re  mind’s  eye,  do  you  see  an  image  of  yourself  in  that  place,  or  do  you 
see  that  place  from  the  same  perspective  as  when  you  were  there? 


Tick 

I  see  myself  there 

I  see  the  place  as  if  I  were  there 

(b)  Think  about  your  present  internal  visual  image  of  that  place,  and  answer  each 
of  the  following  questions: 


Circle  one  item  in  each  row: 


Is  it  in  color  or  monochrome? 

mono 

color 

Is  it  larger  or  smaller  than  real? 

smaller 

about  the  same 

larger 

Is  it  nearer  or  further  than  real? 

nearer 

about  the  same 

further 

Is  it  in  stereo  or  mono? 

mono 

stereo 

Is  there  a  frame  around  it? 

no  frame 

frame 

Is  it  all  around  you  (panoramic)? 

not  panoramic 

panoramic 

Is  it  darker  or  brighter  than  real? 

darker 

about  the  same 

brighter 
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Questionnaire  Part  B 


The  following  questions  relate  to  your  experience  in  your  virtual  reality  session  only. 


BO.  Before  you  entered  the  ‘virtual  reality’  you  were  asked  to  say  ‘Now’  whenever  you 
beearne  aware  of  a  transition  from  the  virtual  environment  to  the  real  world  of  the  lab. 

(a)  If  you  reported  no  or  very  few  sueh  transitions,  what  was  the  reason  for  this? 


Please  tick  against  your 
answer 

1.  I  rarely  experienced  being  ‘in’  the  virtual  world,  and  so 
didn’t  have  much  chance  to  come  back  to  reality. 

2.  I  experience  myself  as  ‘in’  the  virtual  world  almost  all  the 
time,  and  so  rarely  came  back  to  reality. 

3.  I  did  experience  many  transitions  from  virtual  to  real  but 
forgot  to  report  these. 

4.  Other  -  please  explain: 

(b)  If  you  did  make  transitions  from  virtual  to  real,  whether  or  not  you  reported  these  at 
the  time,  what  do  you  remember  as  the  eauses  of  the  transitions?  (For  example,  hearing 
an  unexpeeted  noise  from  the  real  lab  might  eause  such  a  transition). 

Bl.  Please  rate  the  extent  to  which  you  were  aware  of  background  sounds  in  the  real 
classroom  in  which  this  experience  was  actually  taking  place.  Rate  this  on  the  following 
scale  from  1  to  7  (where  for  example  1  means  that  you  were  hardly  aware  at  all  of  the 
background  sounds): 


While  in  the  virtual  reality  I  was  aware  of 
background  sounds  from  the  classroom: 

Please 

tick 

against 

your 

answer 

1.  not  at  all ... 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  very  much  ... 

7 
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B2.  How  dizzy,  sick  or  nauseous  did  you  feel  resulting  from  the  experience,  if  at  all? 
Please  answer  on  the  following  1  to  7  seale. 


/ felt  sick  or  dizzy  or  nauseous  during 
or  as  a  result  of  the  experience... 

Please 

tick 

against 

your 

answer 

1 .  not  at  all 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  very  much  so 

7 

B3.  Please  rate  yowr  sense  of  being  in  the  tank,  on  the  following  scale  from  1  to  7,  where 
7  represents  your  normal  experience  of  being  in  a  place. 


I  had  a  sense  of  "being  there"  in  the  tank: 

Please 

tick 

against 

your 

answer 

1.  not  at  all ... 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  very  much  ... 

7 

B4.  To  what  extent  were  there  times  during  the  experience  when  the  tank  beeame  the 
"reality"  for  you,  and  you  almost  forgot  about  the  "real  world"  of  the  classroom  in  which 
the  whole  experience  was  really  taking  plaee? 


There  were  times  during  the  experience 
when  the  virtual  tank  became  more  real 
for  me  compared  to  the  "real  world"... 

Please 

tick 

against 

your 

answer 

1 .  at  no  time 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  almost  all  of  the  time 

7 
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B5.  When  you  think  back  about  your  experience,  do  you  think  of  the  tank  more  as  images 
that  you  saw,  or  more  as  somewhere  that  you  visited?  Please  answer  on  the  following  1  to 
7  scale. 


The  virtual  tank  seems  to  me  to  be  more 
like... 

Please 

tick 

against 

your 

answer 

1 .  images  that  I  saw 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  tanks  that  I  have  been  in 

7 

B6.  During  the  time  of  the  experience,  which  was  strongest  on  the  whole,  your  sense  of 
being  in  the  tank,  or  of  being  in  the  real  world  of  the  classroom? 


/  had  a  stronger  sense  of  being  in... 

Please 

tick 

against 

your 

answer 

1 .  the  real  world  of  the  elassroom 

1 

2 . 

2 

3 . 

3 

4 . 

4 

5 . 

5 

6 . 

6 

7.  the  virtual  reality  of  the  tank  turret 

7 

APPENDIX  F 


Research  Participant  Information  Questionnaire 
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ID 


Date 


Research  Participant  Information  Questionnaire 

Adapted  from  Finkelstein  (1999) 

Please  fill  in  the  blank  or  eircle  the  appropriate  response. 

1.  What  is  your  age?  _ years  2.  What  is  your  gender?  female  male 

3.  Are  you  in  a  state  of  good  physical  fitness?  yes  no 

4.  How  many  hours  did  you  sleep  last  night?  _ hours 

4a.  Was  your  sleep  sufficient?  yes  no 

5.  Circle  below  the  types  of  medications/substances  you  have  used  in  the  last  24  hours: 

0  -  none 

1  -  sedatives  or  tranquilizers 

2  -  aspirin,  Tylenol,  other  analgesics  or  “pain  killers” 

3  -  anti-histamines 

4  -  decongestants 

5  -  other _ 

6.  Have  you  ever  experienced  motion  or  car  sickness?  yes  no 

7.  How  susceptible  to  motion  or  car  sickness  do  you  feel  you  are? 

0  1  2  3  4  5  6  7 

not  very  average  very 

susceptible  mildly  highly 

8.  Do  you  have  a  good  sense  of  direction?  yes  no 

9.  How  many  hours  a  week  do  you  use  computers?  _ hours  per  week. 
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10.  My  level  of  eonfidenee  in  using  eomputers  is 

1  2  3  4  5 

low  average  high 

11.  1  am _ at  playing  video  and  eomputer  games  (home  or  areade). 

1  2  3  4  5 

bad  average  good 

12.  1  enjoy  playing  video  and  eomputer  games 

1  2  3  4  5 

disagree  unsure  agree 

13.  How  many  hours  a  week  do  you  play  video  or  eomputer  games?  _ hours  per 

week. 

14.  How  many  times  in  the  last  year  have  you  experieneed  a  virtual  reality  game  or 

entertainment?  (CCTT,  immersive  systems)  _ 

15.  Do  you  have  normal  or  eorreeted  to  normal  20/20  vision?  yes  no 

16.  Are  you  color  blind?  yes  no 
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APPENDIX  G 


Scores  and  Personal  Data 
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Table  G-1.  Subject  Personal  Data 


Subject 

VR" 

AGE 

Hours  of  Sleep 

Night  Before  Test 

History  of 

Motion 

Sickness 

Perceived 

Susceptibility  to 

Motion  Sickness'’ 

1 

1 

29 

6 

N 

0 

2 

1 

20 

8 

N 

0 

3 

1 

20 

7.5 

N 

0 

4 

1 

20 

7.5 

N 

2 

5 

1 

21 

6 

N 

0 

6 

1 

18 

7 

N 

0 

7 

1 

20 

7 

Y 

1 

8 

1 

22 

7 

N 

0 

9 

1 

19 

7 

Y 

1 

10 

1 

18 

6 

N 

0 

11 

1 

32 

6 

Y 

1 

12 

1 

19 

5 

N 

0 

13 

1 

19 

6 

N 

0 

14 

1 

23 

5 

N 

0 

15 

1 

18 

8 

N 

0 

16 

1 

22 

7 

N 

0 

17 

0 

32 

5 

N 

0 

18 

0 

19 

6 

Y 

1 

19 

0 

21 

6 

N 

0 

20 

0 

18 

4 

N 

0 

21 

0 

20 

7 

Y 

1 

22 

0 

21 

8 

N 

0 
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Subject 

VR" 

AGE 

Hours  of  Sleep 

Night  Before  Test 

History  of 

Motion 

Sickness 

Perceived 

Susceptibility  to 

Motion  Sickness'’ 

23 

0 

22 

8 

Y 

2 

24 

0 

18 

7 

N 

0 

25 

0 

19 

7 

N 

0 

26 

0 

19 

7 

N 

5 

27 

0 

23 

7 

N 

0 

28 

0 

19 

7.5 

N 

0 

29 

0 

18 

7 

N 

2 

30 

0 

20 

7 

N 

0 

31 

0 

19 

8 

N 

0 

32 

0 

21 

7.5 

N 

0 

Note:  Personal  Data  (Taken  from  completed  Appendix  F.  Participant  Questionnaire) 
‘‘Value  of  1=  Assigned  to  Virtual  Reality  Experimental  group;  0  =  Assigned  to  Desktop 
Simulation  Experimental  Group. 

*’The  subjects  were  asked  to  report  their  perceived  susceptibility  to  motion  sickness  on  a  0 
to  7  scale,  with  0  representing  “not  susceptible,”  and  7  representing  “very  highly” 
susceptible. 
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Table  G-2.  Subject  Personal  Data  (Cont.) 


Subject 

# 

Computer  Use 

(hr/wk) 

Computer  Game 

Skilf 

Computer  Games 

Played  (hr/wk) 

Vision  correctable  to 

20/20? 

1 

5 

4 

1 

ASTIG. 

2 

5 

5 

3 

Y 

3 

27.5 

4 

25 

N 

4 

5 

4 

5 

Y 

5 

4 

5 

8 

Y 

6 

5 

3 

2 

Y 

7 

8 

5 

2 

N 

8 

4 

3 

4 

Y 

9 

5 

5 

2 

Y 

10 

12 

5 

5 

Y 

11 

6 

1 

0 

Y 

12 

11 

4 

2 

Y 

13 

25 

4 

5 

Y 

14 

3 

7.5 

Y 

15 

10 

5 

10 

N 

16 

10 

2 

5 

Y 

17 

2 

1 

0 

N 

18 

1 

4 

0 

Y 

19 

20 

3 

0 

Y 

20 

6 

3 

0 

Y 

21 

5 

3 

1 

Y 

22 

11 

5 

30 

Y 

23 

5 

3 

5 

Y 

24 

11 

3 

2 

Y 
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Subject 

# 

Computer  Use 

(hr/wk) 

Computer  Game 

Skilf 

Computer  Games 

Played  (hr/wk) 

Vision  correctable  to 

20/20? 

25 

10 

5 

8 

N 

26 

5 

10 

Y 

27 

10 

3 

0 

Y 

28 

18 

5 

16 

N 

29 

10 

5 

7 

Y 

30 

8 

5 

4 

Y 

31 

10 

3 

10 

Y 

32 

25 

4 

0 

N 

Note.  ‘‘Subjects  rated  their  own  skill  at  playing  computer  games  on  a  scale  of  1  to  5,  with 
1  representing  “bad”  and  5  representing  “good.” 
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Table  G-3.  Experimental  Raw  Seores  from  Training  Trials 


Subject 

# 

Training 

Trial  1  Time 

Training 

Trial  2  Time 

Training 

Trial  3  Time 

Training 

Trial  1  Error 

Training 

Trial  2  Error 

Training 

Trial  3  Error 

1 

16.30 

11.00 

9.97 

1 

1 

1 

2 

19.42 

11.30 

8.66 

0 

1 

0 

3 

11.18 

8.77 

7.28 

0 

0 

0 

4 

12.15 

7.78 

7.07 

2 

1 

1 

5 

8.47 

7.63 

7.42 

0 

1 

0 

6 

14:24 

9:16 

- 

0 

0 

- 

7 

13:16 

7:49 

- 

2 

0 

- 

8 

12:42 

9:25 

- 

1 

0 

- 

9 

9:46 

9:05 

- 

0 

0 

- 

10 

7:04 

9:12 

- 

0 

0 

- 

11 

15:35 

12:35 

- 

1 

1 

- 

12 

18:05 

8:44 

- 

3 

0 

- 

13 

12:40 

10:11 

- 

2 

2 

- 

14 

18:42 

13:06 

- 

2 

1 

- 

15 

10:55 

8:00 

- 

0 

0 

- 

16 

11:20 

10:04 

- 

1 

2 

- 

17 

85.00 

165.0 

180.0 

17 

1 

0 

18 

100 

- 

- 

2 

- 

- 

19 

100 

75 

- 

23 

3 

- 

20 

112.0 

60.00 

50.00 

12 

5 

4 

21 

42 

23 

- 

0 

0 

- 

22 

32 

- 

- 

0 

- 

23 

34 

33 

37 

0 

0 

8 
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Subject 

# 

Training 

Trial  1  Time 

Training 

Trial  2  Time 

Training 

Trial  3  Time 

Training 

Trial  1  Error 

Training 

Trial  2  Error 

Training 

Trial  3  Error 

24 

18 

39 

39 

0 

0 

5 

25 

31 

23 

- 

0 

0 

- 

26 

72 

23 

77 

4 

0 

9 

27 

26 

- 

- 

8 

- 

- 

28 

25 

- 

- 

6 

- 

- 

29 

32 

- 

- 

14 

- 

- 

30 

99 

26 

- 

6 

0 

- 

31 

77 

32 

- 

0 

12 

- 

32 

35 

- 

- 

10 

- 

- 

Note.  A  denotes  trial  not  executed  or  missing  data 
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Table  G-4.  Questionnaire  and  Spatial  Abilities  Test  Seores 


Subject 

# 

Spatial  Abilities  Test 

Score^ 

Witmer  &  Singer  Presence 

Questionnaire  (Ver.  3.0) 

Avg.  Score 

Slater,  Usoh,  &  Steed 

Presence  Questionnaire 

(1994)  Avg.  Score 

1 

11 

4.65 

2.57 

2 

17 

6.78 

3.14 

3 

22 

6.53 

4.71 

4 

10 

6.49 

4.43 

5 

19 

6.07 

3.71 

6 

18 

4.80 

3.43 

7 

31 

5.21 

5.00 

8 

-6 

3.92 

3.86 

9 

10 

6.19 

4.86 

10 

7 

4.50 

0.00 

11 

12 

5.03 

3.86 

12 

2 

5.66 

4.00 

13 

16 

5.94 

3.14 

14 

21 

6.70 

3.86 

15 

40 

5.80 

5.57 

16 

16 

4.20 

3.57 

17 

5 

1.91 

7.00 

18 

3 

3.51 

3.14 

19 

26 

4.65 

3.29 

20 

12 

4.68 

3.00 

21 

9 

3.44 

0.00 

22 

20 

4.06 

1.86 
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Subject 

# 

Spatial  Abilities  Test 

Score^ 

Witmer  &  Singer  Presence 

Questionnaire  (Ver.  3.0) 

Avg.  Score 

Slater,  Usoh,  &  Steed 

Presence  Questionnaire 

(1994)  Avg.  Score 

23 

19 

0.00 

0.00 

24 

19 

0.00 

1.86 

25 

18 

4.35 

3.00 

26 

18 

0.00 

0.00 

27 

26 

5.14 

4.43 

28 

6 

6.97 

4.00 

29 

10 

5.54 

4.00 

30 

29 

5.20 

5.00 

31 

25 

5.22 

4.57 

32 

26 

4.32 

4.57 

Note.  The  two  presence  questionnaires  ask  several  questions  about  the  subjects’  feelings 
of  immersion  in  their  experienced  environments.  The  subjects  are  asked  to  rate  their 
feelings  on  a  scale  of  1  to  7.  The  score  of  some  questions  is  reversed  to  equate  that  a 
reply  of  “7”  means  a  higher  level  of  experienced  presence. 

“The  Spatial  Abilities  Test  consists  of  42  cube  comparisons  where  the  subject  must 
decide  if  the  cubes  pictured  are  two  different  views  of  the  same  cube,  or  two  different 
cubes.  The  score  is  determined  by  subtracting  the  number  of  incorrect  answers  from  the 
number  of  correct  answers. 
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Table  G-5.  Retention  Test  Scores 


Subject 

# 

Raw  Retetion  Test 

Performance  Time 

Retention  Test  Observed 

Errors 

Adjusted  Retention  Test 

Performance  Time’* 

1 

6.98 

3 

7.02 

2 

6.75 

2 

6.81 

3 

5.07 

3 

6.63 

4 

8 

0 

7.06 

5 

4.45 

3 

6.74 

6 

7.37 

2 

6.77 

7 

7.53 

3 

6.31 

8 

7.5 

3 

7.63 

9 

6.17 

1 

7.06 

10 

4.1 

1 

7.17 

11 

9.22 

4 

6.99 

12 

7.12 

0 

7.34 

13 

5.35 

0 

6.85 

14 

8.38 

3 

6.67 

15 

5.98 

1 

5.99 

16 

8.48 

1 

6.85 

17 

6.45 

1 

7.24 

18 

10.35 

4 

7.31 

19 

6.02 

2 

6.49 

20 

6.02 

1 

6.99 

21 

10.22 

4 

7.09 

22 

8.77 

4 

6.70 

23 

6.06 

0 

6.74 
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Subject 

# 

Raw  Retetion  Test 

Performance  Time 

Retention  Test  Observed 

Errors 

Adjusted  Retention  Test 

Performance  Time'* 

24 

5.53 

0 

6.74 

25 

6 

1 

6.77 

26 

7.15 

2 

6.77 

27 

6.53 

0 

6.49 

28 

5.22 

1 

7.20 

29 

7 

1 

7.06 

30 

5.92 

1 

6.38 

31 

7.7 

0 

6.53 

32 

5.57 

0 

6.49 

Note.  “Times  were  adjusted  with  equation  3,  derived  from  regression  analysis  of  the  raw 
retention  test  performanee  times  and  the  spatial  abilities  test  seores. 


82 


APPENDIX  H 


Recall  Test 
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Figure  FI-1.  Page  1  of  Reeall  Test 


84 


Fire 

Control 

Mode 

Switch 


Panel 

Light 

Brightness 

Knob 


Ammo 

Select 

Switches 


TIS  Ready 
Light 


Gunner's 

Computer 

Display 

Panel 


GPS 

Ballistic 

Door 

Handle 


Gun  Select 
Switch 


Figure  H-2.  Page  2  of  Recall  Test 
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Figure  FI-3.  Page  3  of  Recall  Test 
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Figure  H-4.  Page  4  of  Recall  Test 
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